regular updates

[fenio]

AFAICS it's still being updated by hand from time to time. Would be great to have regular automatic updates of data.

[zbraniecki]

Yeah, so, the main issue is the quality of data.

The data aggregators are very fragile now and every day I'm discovering new workarounds I have to add.

I hope this to stabilize over the next couple days, but with the new data source, I had to workaround the following bugs:

• https://github.com/lazd/coronadatascraper/issues/336
• https://github.com/lazd/coronadatascraper/issues/357
• https://github.com/lazd/coronadatascraper/issues/365
• https://github.com/lazd/coronadatascraper/issues/375

The last of them being quite critical.

The JHU recently removed the "active" and "recovered" cases because they're flaky, and I may have to do the same because at the moment I'm basically projecting them based on previous days and shift in "cases".

I'll investigate data quality tomorrow and try to minimize the amount of "fixes" i have to implement, either by finding new data sources or reducing the coverage. Once the system stabilizes and I have a couple days of no-issues updates, I'll get back to automating it.