Read on Twitter
I find studies like this to be such a headache
On the one hand, I think the research is valid and important
On the other, this headline is SUPER misleading and there are massive caveats not explored in the article https://twitter.com/CNN/status/1295777514976153600
On the one hand, I think the research is valid and important
On the other, this headline is SUPER misleading and there are massive caveats not explored in the article https://twitter.com/CNN/status/1295777514976153600
Study is here: https://www.pnas.org/content/early/2020/08/12/1913405117
This was a pretty simple use of a retrospective dataset. The authors took a large database of births, and looked at whether babies died depending on the race of their physician
It turns out that, when they plugged their numbers into a corrected OLS model, Black babies had a death rate of 601/100,000 compared to white babies at 290/100,000
But if the physician was Black, this reduced to 419/100,000!
But if the physician was Black, this reduced to 419/100,000!
Now, you'll note that this doesn't really match the headline. That's because the headline combined the *unadjusted* results with the *adjusted* results
That's problematic!
That's problematic!
It also reports only relative risk differences, but here I think the absolute risk difference is really informative! The rate is going from 0.3% up to 0.6%, which is a bit less terrifying in context
Delving deeper into the paper tho, there are more caveats
Firstly, physician race. The dataset didn't contain this information...so how did the researchers get it?
Firstly, physician race. The dataset didn't contain this information...so how did the researchers get it?
Well, it turns out they used a lengthy but still not ideal process that can be summed up simply: they Googled the physician names and looked at their photos to define race
Even with the controls described here, that has issues
Even with the controls described here, that has issues
Moving on, there are other worrying limitations here. The dataset contained 4.5 million births, but the primary analysis only uses 1.8 million
What's happening here?
What's happening here?
Well, turns out they had ANOTHER primary analysis - Latino babies treated by Latino physicians
The results might be a bit of a surprise!
The results might be a bit of a surprise!
If you read those regression results, compared to the 0.3% death rate of white babies, Latino babies have a death rate of 0.35%
But this INCREASES to 0.4% when the physician is Latino!
But this INCREASES to 0.4% when the physician is Latino!
Now, the statistical significance disappears in one of the models there, and I'm not saying that it's likely that Latino physicians are killing babies, but it underscores the difficulty of these analyses
Causal attribution is NOT EASY
Causal attribution is NOT EASY
For one thing, the analysis didn't adjust for a few confounders that could be important:
- maternal age
- paternal factors
- BMI/blood sugar/blood pressure (they did adjust for ICD-10 codes but these are pretty blunt)
- maternal age
- paternal factors
- BMI/blood sugar/blood pressure (they did adjust for ICD-10 codes but these are pretty blunt)
Now all of this is not to say that this study isn't worthwhile, important, and should prompt further research
But I think it's worth noting that the CNN headline at the start of this thread that went mega-viral is misleading at best
But I think it's worth noting that the CNN headline at the start of this thread that went mega-viral is misleading at best
If NOTHING ELSE, this could read "0.3% more likely to die"
Anyway, the point here is not to bash the study, but to point out that you should be skeptical of everything ESPECIALLY if it agrees with you because that's where you're most blind
