Read on Twitter
As cases exploded in the U.S., I kept discounting them in my mind: "Well, treatments have improved. Maybe the death rate won't be nearly as high as the summer."
Then I saw this from @trvrb, who sounded the alarm early about COVID-19 in the U.S. https://twitter.com/trvrb/status/1326404894954283008?s=20
Then I saw this from @trvrb, who sounded the alarm early about COVID-19 in the U.S. https://twitter.com/trvrb/status/1326404894954283008?s=20
Shakily, I looked into what Trevor did. He looked at state case and death numbers to determine how many days in the past cases best aligned with today's death toll. Empirically, the number was about 22 days.
Here's the match.
(I'll be using charts I made to illustrate here.)
Here's the match.
(I'll be using charts I made to illustrate here.)
Then, Bedford calculated what he called the "lagged case fatality rate" using that time-shift. In the early days of the pandemic, testing was so limited that the relationship of lab-confirmed cases to deaths is sort of nonsensical. But by summer, the numbers were settling down.
Unfortunately, the lagged CFR settled down at between 1.5% and 1.8%. That's a huge improvement from spring! But there has been almost no improvement since early August. And that's incredibly important and very grim.
If 1.5% of yesterday's average cases (157k) die in 22 days... That's more than 2300 deaths. We're talking about spring-level death numbers. And not in months, but just a few weeks.
If the relationship @trvrb identified holds just through December 9, this is what we're looking at. On my computer, this file is called: The Chart.
When I first saw this analysis, I desperately wanted it to have some key flaw. So, I went to Ryan Tibshirani, who works with Carnegie Mellon's DELPHI forecasting group, which has been named a center of excellence in this field by the CDC. I sent him the numbers and said: HELP.
And he really, really came through with an impressive in-depth analysis, which you can read in full here:
https://htmlpreview.github.io/?https://github.com/cmu-delphi/covidcast-modeling/blob/cfr-analysis/cfr_analysis/cfr_analysis.html#Ensemble_comparison
https://htmlpreview.github.io/?https://github.com/cmu-delphi/covidcast-modeling/blob/cfr-analysis/cfr_analysis/cfr_analysis.html#Ensemble_comparison
He added many caveats to Bedford's model. For example, maybe the lag is a little shorter and the lagged CFR a little lower. There's a ton of noise and uncertainty at the state level. He also pointed out that the CDC's then-current ensemble model was predicting many fewer deaths.
Tibshirani's analysis also showed that if the case report to death report lag was really short— say, 10 days—then the lagged CFR would not only be lower, but it would be showing improvement in recent weeks.
It also suggested 16 days might be an interesting lag to explore. The best-fit CFR there is 1.4%, which would be overall good news.
But Tibshirani also showed that Bedford's simple model — with reasonable parameters put in — actually outperformed the CDC ensemble forecast. (Lower on the chart is better)
Tibshirani's caveat below explains why this is not quite fair to the ensemble, but still.
Tibshirani's caveat below explains why this is not quite fair to the ensemble, but still.
As @whet and I were working on the story for @TheAtlantic, the CDC forecasters released an update. It showed major upward movement in the prediction of deaths in the coming weeks with the top-end of the cone of prediction soaring. https://covid19forecasthub.org/
Then, yesterday, states reported 1,869 deaths, the most since MAY 7. This was a major jump over last Wednesday, but it was very close to the ~1,900 deaths that the most realistic Bedford-style model was predicting (22-day lag, 1.7% lagged CFR).
I have seen a lot of models fail. I wanted this one to have some obvious set of problems that would invalidate what thinking with it suggests. But I couldn't find anything, even with big help, that truly destabilized @trvrb's findings.
This doesn't even account for the downside risk built into healthcare systems collapsing or increased transmission from Thanksgiving and other holidays.
https://www.theatlantic.com/health/archive/2020/11/third-surge-breaking-healthcare-workers/617091/ https://www.theatlantic.com/health/archive/2020/11/pandemic-thanksgiving/617119/
https://www.theatlantic.com/health/archive/2020/11/third-surge-breaking-healthcare-workers/617091/ https://www.theatlantic.com/health/archive/2020/11/pandemic-thanksgiving/617119/
There are a lot of uncertainties and specific numbers may fluctuate, but this analysis has led me to a sad conclusion that I hope proves incorrect:
we are going to see spring-level deaths in December, just weeks before vaccines begin rolling out. It's heartbreaking.
we are going to see spring-level deaths in December, just weeks before vaccines begin rolling out. It's heartbreaking.
You can see the full story of this analysis @TheAtlantic: https://www.theatlantic.com/science/archive/2020/11/coronavirus-death-rate-third-surge/617150/
