Read on Twitter
The "Imperial group" Flaxman et al paper in Nature is the scientific corner-stone justifying "lock down" public health policies.
As that paper says many and strict NPIs are better than few/no NPIs.
As that paper says many and strict NPIs are better than few/no NPIs.
Nature has published a responding meta-science paper - i.e., science to validate science - that finds the Flaxman model shows the *opposite*, that more NPIs had near _no_ effect. https://www.nature.com/articles/s41586-020-3025-y
How is this possible, that the Flaxman model could both find more NPIs were better AND find that more NPIs had little/no effect?
The answer is: The Flaxman model _special-cased Sweden_. It treated Sweden differently to the other countries, in order to produce that result!
The answer is: The Flaxman model _special-cased Sweden_. It treated Sweden differently to the other countries, in order to produce that result!
“the estimated effects of NPIs change markedly when the model is not allowed to give the Swedish data the special treatment that the country-specific last NPI parameter enables.”
With the 'tweak' in place: “Notably, the estimated effectiveness of the public events ban in Sweden is comparable to that of lockdown in the 10 countries in which one was implemented."
“The result above—that is, the public events ban and the lockdown being mutually effective in Sweden and 10 other European countries—was not addressed by Flaxman et al, which is noteworthy as this result undermines the conclusion of lockdown being especially effective.”
So how should we view the Flaxman work now? The responders say: “we suggest that the model, and its conclusion that all NPIs apart from lockdown have been of low effectiveness, should be treated with caution with regard to policy-making decisions.”
How do explain how the Flaxman group came up with a model that manages to effectively special-case Sweden (or equivalently special case "lock-down" in all the others)?
The responders are kind, and say:
“We conclude that the model1,3 is in effect too flexible, and therefore allows the data to be explained in various ways. … This kind of error—mistaking assumptions for conclusions—is easy to make, and not especially easy to catch …”.
“We conclude that the model1,3 is in effect too flexible, and therefore allows the data to be explained in various ways. … This kind of error—mistaking assumptions for conclusions—is easy to make, and not especially easy to catch …”.
Others might ask how the Flaxman group did not test that country-specific 'tweak' on the data they had and knew the effect it had. Or indeed, whether that 'tweak' was added in response to an earlier model not producing the results they wanted.
The former would be sloppy science. The latter would be scientific dishonest. The range of possibilities lie on that spectrum.
And that is what forms the scientific corner-stone justifying these "lockdown" public health policies.
And that is what forms the scientific corner-stone justifying these "lockdown" public health policies.
/fin
