I have a very strong opinion about this paper that went viral over the weekend. I cooled down a bit since coming across it yesterday and will try to refrain from strong wording. Here is my negative post-pub peer-review THREAD
1/

The paper is published in @JAMANetworkOpen, DOI: 10.1001/jamanetworkopen.2020.28786
Have a look to check and form you own opinion
/2

Today @GidMK composed a nice thread discussing the multiple shortcomings of the paper. Yet somehow the elephant in the room was not mentioned. I'm going to focus only on the biggest issue.
/3 https://twitter.com/GidMK/status/1327872428744183809

The biggest issue is the assumed *deterministic* relationship between years of schooling and life expectancy. The authors estimate the lost years of schooling and translate them directly into lost years of life.
/4

the relevant bit of the Methods part. Using 7 economic studies (see in the next tweet) that quantify the association between years of schooling and death rates, the authors arrive at the 0.75 estimate of the relative risk of annual death per one additional year of schooling
/5

The core *deterministic* ratio is derived from these 7 papers, just note that none of them are from demographic journals.
/6

...sigh...
Let's pause a bit and appreciate how big is the 25% reduction in age-specific death rates.
[how about giving everybody several more years of schooling and becoming immortal? or do people without education just die right away?]
/7

If we take the 2017 death rates for the US males via @HMDatabase and reduce them by 25% the life expectancy jumps from 76.3 to 79.9.
And that's the effect attributed to just one additional year of schooling.
/8

The real differences in educational attainment between large groups of population are pretty big. In 2010 among non-hispanic white males in the US there were about the same number of deaths for people with 16+ and <11 years of schooling. That's a solid 5+ years difference
/9

The above table comes from a great paper by Isaac Sasson, 2016, Demography, 269–293.
Trends in Life Expectancy and Lifespan Variation by Educational Attainment: United States, 1990–2010. https://doi.org/10.1007/s13524-015-0453-7
/10

Of course differences in longevity by educational attainment are pretty large. Here is another figure from Sasson 2016
/11

Education is very important. No doubt about it. Years of schooling accommodate various other effects in the association with longevity. It's a very simple a good proxy for both SES and various lifestyle covariates.
/12

But it does not explain *all* the variance in longevity, which is the implicit assumption of the *deterministic* 1 to 1 translation used in this @JAMANetworkOpen paper.
/13

This association between educational attainment and longevity can't be used to directly translate ~2 months of lost education into ~2–3 months of lost expected years of life. Here are the core results of the paper again.
/14

So this was the elephant in the room
It's the cornerstone idea of the paper that lost schooling years can be directly translated into lost years of life. And it is wrong.
/15

Based on these flawed calculations, the authors proceed to an even more questionable comparison of the YLL due to the lockdown school closures and YLL due to the direct COVID19 deaths through 30 May 2020. ~25m kids aged 5–11 vs 88k c19 deaths of the ongoing pandemic. Really?
/16

There are many more issues with the assumptions and the analysis. See many of them, in the linked thread by @GidMK
/17

Through these very problematic (to frame it nicely) assumptions and correspondingly shaky results the authors arrive at the very big conclusions claiming that their paper provided sufficient evidence to keep the schools open. It did not.
/18

Such policy recommendations are
downright irresponsible
and, if heard and implemented, are
very dangerous
/19

I need to mention: I came across this paper featured by @ProfEmilyOster (61.2k followers) and re-tweeted non-critically by @pengzell who wrote a brilliant paper w/ @MarkDVerhagen and @ArunFrey on the drop in school performance due to the pandemic
/20 https://twitter.com/pengzell/status/1322796588746575872

It would be nice to hear your thoughts on my critical review. I'd also appreciate to hear from the authors:
@DAChristakis @drwilvancleve @Zimmermanfred
Especially since the main author considers the piece one of the most important work ever
/21 FIN https://twitter.com/DAChristakis/status/1326922136095617024

Here is a bonus from eSupplement. This is how the crucial risk ratio of 0.75 was estimated from the 7 studies. Am I the only one seeing this averaging weird: 5 EU studies w/ odds close to 1 + 2 US studies w/ broad uncertainty = 0.75 ???
asking opinion of meta- researchers
/22