Read on Twitter
A thread about trying (and failing) to notify JAMA about an incorrect COVID-19 study that they published. 
1/N
1/N
The initial article (a research letter) claimed that stay-at-home orders slowed the growth of COVID-19 hospitalizations in 4 states. 2/N https://jamanetwork.com/journals/jama/fullarticle/2766673
The authors ran an egregiously incorrect extrapolation exercise that vastly and absurdly overstated hospitalizations in the absence of stay-at-home orders.
This is not statistical arcana, it's very obvious once you think about it.
3/N
This is not statistical arcana, it's very obvious once you think about it.
3/N
I wrote a twitter thread about this:
https://twitter.com/paulnovosad/status/1265793481387249679?lang=en
Muffadal Baxamusa and @JohnASpry had noted the same error and invited me to join in a much clearer response they were writing to JAMA. 4/N
https://twitter.com/paulnovosad/status/1265793481387249679?lang=en
Muffadal Baxamusa and @JohnASpry had noted the same error and invited me to join in a much clearer response they were writing to JAMA. 4/N
A few other people wrote me after my thread, saying, "you need to respond to this! Science can't make progress if nobody does anything when errors are not corrected." 5/N
We wrote a response letter showing that the authors' method would have found a drastically declining hospitalization rate even in 3 states that did not have stay-at-home orders. You can read our letter here: https://medium.com/@johnspry/disregard-the-claims-of-association-of-stay-at-home-orders-with-covid-19-hospitalizations-in-4-bd5247c56b3 6/N
The JAMA editor (same one who handled the original incorrect paper) sent a dismissive form letter, "The manuscript is really a comment on the study, so you should leave a comment on the web site." 7/N
It was a little disappointing but ok fine, editors are busy. So we wrote the letter up as a medium post and went to leave a comment on the JAMA article web site. 8/N
The comment on the JAMA web site was rejected by the editors without explanation.
The comment on the JAMA web site was rejected by the editors without explanation.
In fairness to the editor, an earlier comment post made a similar statistical point but with no evidence about the placebo test. 9/N
The authors responded and doubled down, saying that their article passed a placebo test in South Dakota (which of course they declined to show). This is extremely hard to believe— here are placebo tests in 3 states *without* stay-at-home orders: 10/N
Giving the authors the final word is fine, but when the final word is flagrantly misrepresenting the evidence...
11/N
11/N
Sidenote 1: We aren't saying that stay-at-home orders don't work. We are saying JAMA shouldn't publish articles with zero statistical validity, regardless of the conclusions, and that journals/authors should admit when they are wrong. 12/N
Sidenote 2: If the paper concluded that stay-at-home orders had no effect, do you think JAMA would have published the article and rejected all criticism of it? 13/N
This particular case is not a big deal— it's not the world's most influential article.
The big deal is this culture around not admitting mistakes, making it hard for mistakes to be brought to light, doubling down instead of admitting error. 14/N
The big deal is this culture around not admitting mistakes, making it hard for mistakes to be brought to light, doubling down instead of admitting error. 14/N
I understand editors are under a lot of pressure to rapidly publish COVID-19 material and that some mistakes will be made. But science proceeds by noting its errors. 15/N
I was curious when we started this process whether it would be possible or worthwhile to get a medical journal to acknowledge a deeply erroneous article on a hugely important public policy issue.
In this case at least, the answer is no. 16/16
In this case at least, the answer is no. 16/16
