Read on Twitter
New preprint: Group Sequential Designs: A Tutorial https://psyarxiv.com/x4azm Friedrich Pahlke, Gernot Wassmer, and myself explain how to design and report sequential designs using rpact. The rpact shiny app is a game-changer. It makes it very easy to implement sequential designs.
It comes with a step-by-step vignette: https://github.com/Lakens/sequential_tutorial/blob/main/vignette/vignette.pdf I am quite excited about this article – I think it might finally allow psychologists to benefit from more efficient sequential designs that are common in other fields.
You've probably heard of sequential designs, as tests of Corona vaccines report results from interim analyses. Psychologists intuitively realized interim analyses are efficient. But they performed interim analyses without correcting the alpha level (known as ‘optional stopping’).
That is not good practice. Now, many think all interim analyses are p-hacking. That’s not true. You just need to correct the overall Type 1 error. One valid way is the simple Bonferroni correction. But you can be more efficient, and more flexible.
When I first wrote about sequential analyses in 2014, I think it was a bit too early. Psychologists were less familiar with R. Did Shiny even exist? Preregistration (highly recommended due to the additional flexibility sequential analyses provide) was new for most people.
But now, I think psychologists are better prepared. They are used to Shiny apps and R. Preregistration is not new. And even equivalence testing (which allows 'stopping for futility' in sequential designs) is becoming known more widely.
There are alternatives to group sequential designs. @MSchnuerch shows ( https://psycnet.apa.org/record/2019-52380-001) the sequential ratio probability test is superior to both group sequential designs and sequential Bayes factor designs if you can test after every participant.
Group sequential designs were developed for when it is logistically difficult or too costly to analyze data after every participant. When you can analyze after every participant, you should use the sequential ratio probability test – it is better than all alternative approaches.
But since you get the biggest boost in efficiency from adding the first few looks at the data, and because correcting the alpha level for a few extra looks is so easy and gives you so much more efficiency, group sequential designs might more easily fit in your workflow.
This paper is a complement to my recent preprint on sample size justification https://twitter.com/lakens/status/1346003781935357952?s=20 where I often recommend sequential analyses (e.g., if there is large uncertainty about effect sizes). I hope sequential analyses will make you a more efficient researcher!
