I have no quality memes to add, but I did read through this paper and want to talk about an aspect of the proposed metric that I find really compelling and that I think might get lost in the shuffle.

First, here's @economeager's on the paper: 1/ https://twitter.com/economeager/status/1334191213306654720

This paper has everything, including the name of my mathcore band's first album 2/

A lot of the talk (read: memes) I've seen has focused on the big bad of changing results / losing significance when even small parts of the sample are dropped. And, yeah, that's not fun! 3/

But what I really like about this metric is that it offers a way to consider how the generalizability of our results relates to our sample. Even "perfect" samples may have a situatedness that defies general policy prescriptions. 4/

Reading between the lines ( @economeager can correct me), it seems this metric is useful even in the context of a "sample" that really is a census of a temporally stable population. What if your ATE is really sensitive to the removal of the influential set of obs? 5/

Then maybe your general intervention is actually more targeted than you think it is! That's useful to know! One can then conduct other analyses of heterogeneity or use more qualitative methods to see what's going on. 6/

In my area of higher education policy, understanding how our sample affects our results and policy prescriptions is vitally important: poor policies waste already limited resources and usually hurt those worst who have the least opportunity to start with. 7/

Anyway, I'm going to keep mulling this over and, when I'm feeling brave/foolhardy, will run this test on one of my old papers—I'll also give it a whirl in the future. I would encourage my other edquant people to take a look. 8/8