Read on Twitter
As far as I know, the Journal of Politics is the first to have the policy that "The submission of unregistered laboratory, field, and survey experiments will not be accepted." If you've followed my work, you will not be surprised to hear I have mixed feelings about this. https://twitter.com/The_JOP/status/1350385896315617280
Pro: I think it makes perfect sense to require preregistration of a carefully planned study. But: A good test of a theory requires a lot of preparatory work on the auxiliary assumptions, and establishing a strong derivation chain. (see our recent paper https://journals.sagepub.com/doi/pdf/10.1177/1745691620966795).
Against: Part of such preparatory work might involve exploratory experimentation. This is valuable work, and I wouldn’t know why a journal would not want to publish this (as long as it is clearly described as such).
Pro: It will make it clear which tests were planned before looking at the data, and which after. This makes it possible for people to transparently evaluate the severity of each test – this is the value of preregistration. http://team1mile.com/sjpr62-3/wp-content/uploads/2020/03/Lakens_JPR623221-230.pdf
Against: The risk is you force people to write meaningless preregistrations when they have no strong prediction, just to jump through a hoop. People will write them, but they will be a distraction at best, and used as a heuristic at worst.
Precondition: People will need to quickly learn how to write a good preregistration. @LisaDeBruine and I worked on what a gold standard could look like, making the hypothesis test machine readable: https://psyarxiv.com/5xcda/ (it’s doable, but not trivial).
So in the end, whether this works out will depend a lot on how it is implemented, and the instructions reviewers get. Preregistration does not necessarily improve the quality of a study.
In the end, as @wolfvanpaemel writes ( https://doi.org/10.1007/s42113-019-00056-9) we want reviewers and readers to evaluate the severity of the test. If they do this carefully, access to the preregistration can help. Let’s hope it does.
