Read on Twitter
A new paper has been published by John Ioannidis and Jay "Great Barrington Declaration" Bhattacharya on "lockdowns" as a COVID-19 preventative measure
Let's do some twitter peer-review! 1/n https://twitter.com/andrewbostom/status/1348271129887191041
Let's do some twitter peer-review! 1/n https://twitter.com/andrewbostom/status/1348271129887191041
2/n The paper is here, and it's an interesting read:
https://onlinelibrary.wiley.com/doi/epdf/10.1111/eci.13484
https://onlinelibrary.wiley.com/doi/epdf/10.1111/eci.13484
3/n The paper takes 10 countries' worth of data, and compares their COVID-19 case numbers against the restrictions they had in place in early 2020, comparing those with less-restrictive non-pharmaceutical interventions (lrNPIs) with more-restrictive NPIs (mrNPIs)
4/n The basic findings are that, compared to South Korea and Sweden, the marginal impact of implementing mrNPIs was not possible to discern in this study
In other words, "lockdowns" don't have a significant marginal benefit over a suite of other measures
In other words, "lockdowns" don't have a significant marginal benefit over a suite of other measures
5/n This has actually been shown before, using much larger datasets and more rigorous (but still imperfect) analyses, so it's a bit odd that this particular paper has been seized on so much https://science.sciencemag.org/content/early/2020/12/15/science.abd9338
6/n I should also say I have something of an intellectual conflict of interest here, because the authors conclude something that I've been saying since March - the interventions pursued may be less important than the way that they are implemented
7/n All that being said, what's the science here
Well, it's a bit...lacklustre really
Well, it's a bit...lacklustre really
8/n The sample size is minuscule. 8 mrNPI and 2 lnNPI countries is far too few to make any realistic conclusions about much, especially given how wildly different these countries are
9/n The authors claim that they used these countries because those were the only ones where they could get data for each administrative region, but if that's the case then they really can't have looked very hard
10/n Off the top of my head, I can name at least half a dozen other countries for which case/policy data is available by day in every administrative region. Switzerland, Australia, New Zealand, China, Brazil, etc
11/n Indeed, going by the references that the authors cite, this issue may have arisen because they used Statista as their primary source for case data, which is...not ideal
12/n The definition of restrictions is also a bit weird. I mean, South Korea didn't forcibly close businesses, but they do have national legislation allowing practices that many countries would find very restrictive (as the references the authors cite show)
13/n There's also nothing in this paper about lags for any policy for implementation, how policies were associated with dates etc
That's a huge issue!
That's a huge issue!
14/n We know that the lag between policy introduction, implementation, and outcome is not immediate, and this is likely to vary by country, so simply comparing them day-by-day as this paper appears to doesn't really give us any indication of their impact
15/n Furthermore, the lrNPIs themselves are really poorly elucidated. This is FAR from a fair summation of the complex and detailed work South Korea put in to controlling COVID-19!
16/n I mean, reading the paper you might get the impression that all South Korea did was some optional social distancing, emergency declaration, and case quarantine, rather than a coordinated and multi-step approach including HUGE healthcare investment
17/n There's also not much effort to disentangle the complexities of the marginal benefit of each intervention, unlike previous research. It's likely, for example, that closing schools in Sweden (that did little else) had a huge impact...
18/n ...but that this was not as effective as in Italy, which had many interventions
19/n The authors also use some fairly inappropriate causal language throughout. These are the potential benefits ASSOCIATED WITH the announcement of policies in each place, we certainly can't infer a causal impact here
20/n In other words, there are innumerable confounding factors that may have made the interventions more/less effective, like the age structure of the population, how socially distanced they were pre-pandemic etc...
21/n At best, this study provides us with some evidence that mrNPIs are not associated with a large marginal benefit in terms of case numbers over lnNPIs, when comparing a tiny group of dissimilar nations
22/n More realistically, I think we can probably say that the paper tells us little useful except that analysing the impact of NPIs generally is extremely hard
23/n This is a bit of a shame, because I actually agree wholeheartedly with the authors that there is a cost to restrictive NPIs, and the marginal benefit of (say) stay-at-home orders is likely to be quite small in many circumstances
24/n That being said, this paper just doesn't tell us anything useful about these mrNPIs beyond some more very vague evidence that they may not be as beneficial on top of other interventions (maybe)
25/n Ultimately, the authors may have failed to find a benefit of business closures or stay-at-home orders, but the methodology used just doesn't give us enough information to say much, if anything, conclusively
26/n Some more issues with the study, which gets worse and worse the more you look at it! https://twitter.com/AndreasShrugged/status/1349464781145731073?s=20
