Read on Twitter
How can COS claim to improve rigour, when it systematically ignores critiques of proposed "rigorous" "solutions" (like preregistration) from among the most rigorous research traditions in psychology, such as computational cogsci & mathematical psychology?
It's interesting. The clip repeats standard narrative of incentive structure & incomplete reporting. Yet, I wonder what incentives motivates some OS advocates to ignore rigorous critique of their views. Is acting as if such critiques do not exist not also selective reporting?
If you bump into these tweets, and think "huh, what critiques??", let me link you some below
1. "The case for formal methodology in scientific reform", by @zerdeve et al. https://www.biorxiv.org/content/10.1101/2020.04.26.048306v2
2. Related to 1., see also this talk: )
3. "Is Preregistration Worthwhile?", by Szollosi et al. https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(19)30285-2?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1364661319302852%3Fshowall%3Dtrue
4. Related to 3. see also this talk: https://twitter.com/IrisVanRooij/status/1338165779263598594?s=20
5. "Psychological science needs theory development before preregistration", by moi https://featuredcontent.psychonomic.org/psychological-science-needs-theory-development-before-preregistration/
6. Related to 5. see also this talk: https://twitter.com/IrisVanRooij/status/1334197623880867848?s=20
7. "Preregistration of modeling exercises may not be useful", by N. MacEachern & @TrishaVZ https://link.springer.com/article/10.1007/s42113-019-00038-x
8. "Paths in strange spaces: A comment on preregistration", by @djnavarro https://psyarxiv.com/wxn58
9. "Cultural problems cannot be solved with technical solutions alone", by @simonlilburn et al. https://link.springer.com/article/10.1007%2Fs42113-019-00036-z
10. "Between the devil and the deep blue sea: Tensions between scientific judgement and statistical model selection", by @djnavarro https://link.springer.com/article/10.1007/s42113-018-0019-z
11. Related to 10, see also this talk: https://twitter.com/IrisVanRooij/status/1109557573345308672?s=20
(Intermezzo: Some of these can read as direct critiques of dominant open science / science reform narratives, but the underlying theme is a fundamental difference in views on scientific inference / epistemology. The papers linked below also spell this out more explicitly)
12. "How computational modeling can force theory building in psychological science" (in press, Perspectives on Psychological Science), by @o_guest and @andrea_e_martin
Preprint: https://psyarxiv.com/rybh9/
Preprint: https://psyarxiv.com/rybh9/
13. Related to 12., see also this talk:
14. "Theory before the test: How to build high-verisimilitude explanatory theories in psychological science", by @giosuebaggio and myself https://journals.sagepub.com/doi/full/10.1177/1745691620970604
15. "Theory development requires an epistemological sea change", by @giosuebaggio and myself https://www.tandfonline.com/doi/full/10.1080/1047840X.2020.1853477
16. "If mathematical psychology did not exist we might need to invent it" (in press, Perspectives on Psychological Science), by @djnavarro https://psyarxiv.com/ygbjp/
17. "Scientific discovery in a model-centric framework: Reproducibility, innovation, and epistemic diversity", by @zerdeve et al. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0216125
(Thread far from finished; will continue shortly)
18. âArrested theory development: The misguided distinction between exploratory and confirmatory researchâ, by Szollosi & Donkin https://psyarxiv.com/suzej/
Not as recent but relevant to neglected epistemological views in the standard open science narrative:
19. âInference to the best explanation: A neglected approach to theory appraisal in psychologyâ, by @BrianHaig https://twitter.com/IrisVanRooij/status/1322280548769943558
19. âInference to the best explanation: A neglected approach to theory appraisal in psychologyâ, by @BrianHaig https://twitter.com/IrisVanRooij/status/1322280548769943558
20. âPsychologists psychologizing scientific psychology: An epistemological reading of the replication crisis,â by @ivanflis https://journals.sagepub.com/doi/full/10.1177/0959354319835322
Let me also add a few more methodological critiques:
21. âDoes preregistration improve credibility of research findings?â, by @RubinPsyc https://drive.google.com/file/d/16e4cUDJQ_ZhFxp18OygLyt-uMprbOd4z/view
21. âDoes preregistration improve credibility of research findings?â, by @RubinPsyc https://drive.google.com/file/d/16e4cUDJQ_ZhFxp18OygLyt-uMprbOd4z/view
22. âUnderstanding replication in a way that is true to scienceâ, by @BrianHaig https://psyarxiv.com/v784s/
(For some of the papers linked here you can find thread summaries via https://metatheorist.com/Now-Reading/ and others are still in preparation)
I should note that I am far from the first to make the observation that some OS advocates/sci reformers seem have repurposed the âfile drawerâ to hide away critical and competing views, and Iâm sure I wonât be the last either. https://twitter.com/zerdeve/status/1326248784175620096
I should also note that among the authors referenced in this thread thereâs diversity of views, with a lot of nuances, differences in focus and divergence of opinions and epistemologies. Donât confuse these critical views as a monolith nor mistake âcritiqueâ for âanti-opennessâ.
Case in point: @djnavarro is generally more nuanced than me 
As are many othersâ whose work I linked to. Please read their work to understand their positions, views, arguments and all the nuances. It will be worth it. I promise. https://twitter.com/djnavarro/status/1349470388980338688
As are many othersâ whose work I linked to. Please read their work to understand their positions, views, arguments and all the nuances. It will be worth it. I promise. https://twitter.com/djnavarro/status/1349470388980338688
Just out! By wonderful philosopher Liz Irvine. Let me quickly add it to the thread
23. "The role of replication studies in theory building", by Elizabeth Irvine https://journals.sagepub.com/doi/full/10.1177/1745691620970558
23. "The role of replication studies in theory building", by Elizabeth Irvine https://journals.sagepub.com/doi/full/10.1177/1745691620970558
