Earlier today I spent some time shitting on this paper without a broader discussion of why I was doing so

I shall hold forth on this matter now as I think it is revealing of the absolute state of empirical social science and microeconometrics in particular https://twitter.com/RRHDr/status/1295488617125687297

I am not going to sink into a detailed methodological critique because

1) while I hate their antique methods this is unnecessary to make my point, and

2) I am writing for a general audience and want to demonstrate that any idiot can reject papers that uses such methods badly

heres some text from the methods section along with some of the figures from the paper

depending on your background you may find your eyes glazing over before opening it or you may be shouting imprecations at your phone

good news if your former! it doesnt fucking matter

heres a dumb person version of the Findings

meditate on it for a moment before we continue

"black babies were three times more likely to die if overseen by a white doctor"

really

REALLY

fucking really

@Fixed_Effects is the corresponding author and he may take this thread as my Correspondence

suppose for a moment that this were actually true

what do physical reality do you imagine might cause this outcome?

what specific actions can you imagine all white doctors taking that no black doctors take that would *triple* infant mortality independent of everything else?

stray intrusive thought

'average white doctor kills black babies at triple rate of black doctors' factoid a statistical error. average white doctor kills black babies at same rate. Racist Georg who delivers in a cave and eats over 10,000 each day, is an outlier adn should not h

generally when a model produces fantastical results you should feel good about discarding the model. and this is absolute insanity

there are a million things that can go wrong in this kind of estimation and the approach used by the authors handles like . . . five of them

this kind of cockup is the *norm* in reduced form microeconometrics

it is remarkable only for being wildly unreasonable, and nevertheless making it through a paper-thin review process (by people whos careers are built on the same methods) because it produced the Correct result

the authors are careful to run through a series of apotropaic tests to ensure that a few well-understood issues did not obtain in their model

its thorough, people myself included spend years learning these Rituals

its akin to making sure submarine screen doors are up to code

if you complete the Ceremony it is extremely unlikely that anyone will bother observing that your results are absolutely cockamamie

its econometrics-by-recipe

"order the data in such a way, run all the standard tests, publish, gib tenure"

no iota of reflection necessary

This paper is so egregious that it knocked me awake

If they'd claimed /using the same methods/ like "5% increase in mortality" I would have slept through it

But--those methods would have been equally bad /had they produced a reasonable result/!

I want to emphasize that in closing

Everything is fucked and you don't usually see it because it's not egregious enough to break the surface but everything in social science is fucked.

Don't succumb to Gell-Mann Amnesia just stop reading empirical papers. Ed Prescott was right.

Special thanks to @RRHDr, @Fixed_Effects, @aaronsojourner & @LauraHuangLA for the object lesson

And to @PNASNews, which my old undergrad advisor helpfully explained stands for "Prints Nearly Any Shit"

You should all be embarrassed but I bet you won't be