Read on Twitter
Y'all, this was accepted for publication in Psychological Science today: "Don't ditch the laptop just yet: A direct replication of Mueller and Oppenheimer’s (2014) Study 1 plus mini-meta-analyses across similar studies." https://psyarxiv.com/vqyw6
Compared to taking notes longhand, we found that taking lecture notes with a laptop boosted word count in notes and overlap between the notes and what the lecturer said but it didn't reduce quiz performance.
But our single replication was imperfect; problem was that the data collection setting wasn't tightly controlled. So we found 7 other similar studies to meta-analyze to see if the predicted superiority of longhand note-taking would emerge across them all.
The meta-analyses revealed strong effects on the notes content variables but negligible effects on quiz performance.
Maybe there's something to the nullness.
Or maybe there's a constraint on generality...
Maybe there's something to the nullness.
Or maybe there's a constraint on generality...
Participants in the original Study 1 were more likely to say they typically use a laptop to take notes in class than participants in our replication study in 2017. That may matter.
In the end, this work nicely captures the challenge of trying to figure out what's up when original and replication studies find different results. In that case, as we conclude, "there are three interpretations: 1) there was a problem with the replication;..."
2) "there was a problem with the original research; and 3) the phenomenon under study is not enduring or universal (i.e., there’s a constraint on generality). These interpretations are not mutually exclusive. In fact, all
three apply here. Situation noise was a problem..."
three apply here. Situation noise was a problem..."
"with our replication. Weak evidence (large p value, Bayesian evidence favoring the null hypothesis) and a small sample size were problems with the original study. And a difference in note-taking medium preferences between the two may represent a constraint on generality."
A recent meta-analysis of laptop versus longhand note-taking revealed a small effect supporting longhand superiority in a larger, heterogeneous set of classroom studies (Allen et al. 2020) so this story is still being written.
In fact, I'm keen to run a multi-lab registered report that aims to examine this question using methods that are closer to what happens in the classroom while also retaining experimental control. In fact, I got pretty damn far writing a Stage 1 Registered Report with many others!
I tweeted about the registered report project a couple years ago. https://twitter.com/HeatherUrry/status/970384237147906048?s=20 (Just pulled the paper open; it's like really close to ready! Let me know if you're interested - maybe that'll be the encouragement I need to polish and submit. Gotta get back to it!!)
In the RR, we write: "Missing from the literature at this stage is a powerful study that 1) estimates the extent to which note-taking in general and note-taking medium in particular impact immediate and delayed test performance after studying one’s notes in a way that,..."
"2) allows us to generalize beyond specific sites, students, lectures, and quiz items, and 3) creates a rich set of materials and data that afford exploration of variables that are germane to understanding note-taking effects on performance."
Meanwhile, cheers to the 87 graduate and undergraduate student co-authors on the new paper at Psych Sci, including @ChelseaCrittle. It's rewarding to see this project from over 3 years ago cross the finish line. Preregistration, materials, data, and code: https://osf.io/tr868
