I think of research design (informally) as being fundamentally about whether you have a story for where your identifying variation comes from--contrasted with the case where your identifying variation is basically an unknown residual. https://twitter.com/paulgp/status/1357449139236265986
If your proposal is, "I want to know whether health insurance reduces mortality. Let's compare people who look similar on observables." You don't have a research design. You have no story for why some people are treated and some are not treated.
Alternatively, if you do a diff-in-diff comparing states where an insurance reform was adopted in one state but not another, you at least have a story for why some people were treated and others not.
Now of course, this does not eliminate "back-door" concerns. Maybe one state passed a health insurance reform because they expected that w/o it a bunch of people would die due to a lack of health insurance.
A "clean" research design is one where your story for why some people are treated and some are not gets as close as possible to a randomized experiment. In other words, a good research design isolates variation where confounding stories are very unlikely.
But I use the term, "So and so has a research design" to mean, "So and so has a story that explains the variation in treatment that identifies their effect". Note that you could write a DAG for the "matching" story above -- but that doesn't make it a research design.
RCTs, IV, RD, diff-in-diff are research designs in this sense, but they are not the only such research designs. An IO equilibrium model where marginal costs are used as an instrument for price would also count as a research design in this sense.
Perhaps there is a more formal version of this distinction that would clarify things.
You can follow @Jabaluck.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled:

By continuing to use the site, you are consenting to the use of cookies as explained in our Cookie Policy to improve your experience.