Read on Twitter
This is a good example of why talking about RCT results is often problematic.
Summary here: "participation did not work, NGO program did". But is that so? https://twitter.com/rglenner/status/1361023351616077826
Summary here: "participation did not work, NGO program did". But is that so? https://twitter.com/rglenner/status/1361023351616077826
Let us take a closer look at the full paper ( https://www.nber.org/system/files/working_papers/w14311/w14311.pdf ).
What was the intervention?
Basically, two treatment arms were about "participation". For one, external people came to the village for (at least) two days, providing information. That is it.
What was the intervention?
Basically, two treatment arms were about "participation". For one, external people came to the village for (at least) two days, providing information. That is it.
For the second treatment arm, there was an additional aspect: training village members in learning assessments.
And then, finally, the third treatment arm: training local volunteers in teaching children.
And then, finally, the third treatment arm: training local volunteers in teaching children.
Result: sending people for 2+ days into a village does not create a buzz for active involvement in improving schools measured at the endline a year later.
Is this a surprise? Is this finding insightful and should it be used to guide policy?
Is this a surprise? Is this finding insightful and should it be used to guide policy?
Many people, if told in such plain language, would probably say: how could you expect some 2-day plus intervention to have an impact on a complex social process like citizen demand and collaboration for improving learning? Well, I don't know.
Most people who work on the ground probably would argue that one needs to build relationships with people. One needs to create multi-stakeholder alliances for better schools. One needs to involve the bureaucracy, teachers, and local government. It takes time.
If we agree on this, we need to seriously introspect how such papers can become so popular and influential. And how such misleading summaries distort a public debate at the expense of big, structural, and meaningful system-wide change.
I think RCTs can be useful for some aspects. But they are used far beyond for that they are meaningful. And they are problematic for this reason. We need different evidence to make a meaningful difference.
