Read on Twitter
I'm proud to have contributed to this paper on community forecasts of replication rates. Most credit goes to @michaelbg01 and @DViganola and also to ALL my @ReplicationMkts colleagues. A few thoughts below: 1/8 https://twitter.com/michaelbg01/status/1286098900403621889
Replication isn't everything.
* Not every good study will replicate successfully every time (see sampling/measurement variation).
* Plenty of flawed studies are highly likely to replicate (see overbroad interpretation of results). 2/8
* Not every good study will replicate successfully every time (see sampling/measurement variation).
* Plenty of flawed studies are highly likely to replicate (see overbroad interpretation of results). 2/8
Nonetheless, replication is crucial. The low rate of replication attempts, and lower rate of replication success, should be treated as a huge problem in the social sciences and beyond. Many are still in denial. 3/8
Our @royalscience #OpenScience paper ( https://royalsocietypublishing.org/doi/10.1098/rsos.200566) doesn't present replication rates, just community expectations of replication rates - but community expectations are important. 4/8
Our follow up research will include actual replication results showing when beliefs about replication rates were right, or wrong, and offer much more detail as we shift attention from forecasting aggregate rates to forecasting individual study-claims chance of replication. 5/8
We'd benefit (and Science would too!) if *you* share your judgment about how likely study-claims are to replicate successfully. Sign-up before the prize money is all won ;) https://www.replicationmarkets.com/ 6/8
Thanks again to all my colleagues on this @ReplicationMkts project including co-PI: @ctwardy @flxhlzmstr @DViganola @michaelbg01 @pretzelsbaby @McCannLynda @yangl1u and several crucial coauthors who, perhaps wisely, keep off Twitter. 7/8
And we all stand on the shoulders of giants like @PTetlock who proved the power of crowdsourced forecasting, and @BrianNosek who helped pioneer large scale systematic replication efforts, and many many more who are contributing to the movement to improve science. 8/8
