Read on Twitter
We're thrilled to announce that the latest addition to the "Sherlock papers" family is now live! This was a super cool paper to work on, and it builds on awesome science from @kanileke, @ChrisBaldassano, @ptoncompmemlab, @HassonLab, and others.
... https://twitter.com/NatureHumBehav/status/1359905895862456324
... https://twitter.com/NatureHumBehav/status/1359905895862456324
We developed a text embedding model for turning video annotations and recall transcripts into geometric shapes. When we looked at those shapes, all sorts of neat patterns started to emerge, and we gained some new insights into how our experiences might be encoded into memories.
We first noticed that the way people recounted a Sherlock episode they had watched contained the same sorts of "event boundary" patterns that @ChrisBaldassano et al. discovered in people's neural responses to the same show. Kind of a nice brain-behavior connection.
We also noticed that only certain aspects of the episode's shape were consistently reproduced in people's memories. After some digging, we discovered that people all tend to remember high-level plot points, but they varied in their memory for low-level details.
On one hand, it seemed intuitive that our memory systems should prioritize retaining the basic narrative over non-essential details. But then we thought more about the implications of this. They are...profound.
To give some context, the way researchers typically study memory uses super simple (often awkward) tasks, like memorizing random word lists. Most modern memory theories are directly inspired by these sorts of tasks. But these tasks don't *have* essential details.
Aside: @roelwillemsRU, @samnastase, and @branka_mvojevic wrote an awesome paper arguing this point and others in more detail; you should check it out: https://tinyurl.com/1b4b1jpg
The finding that our memory systems prioritize high vs. low level info differently, and that priority for low level info is idiosyncratic across people is also neat. It's one reason why people with a shared experience might have totally different takeaways.
Using our framework, we conceptualize experiences as shapes that are "transformed" (geometrically) by our memory systems into new shapes. But each person has a unique transformation function that determines what they'll remember and how their memory will distort reality.
We wondered when these supposed transformations and distortions might be happening. Is it during the experience itself? Or are those changes introduced only once we try to *remember* an experience?
We used our model to look at how people's brains responded as they were watching the episode. We found something really cool.
The anterior temporal system seemed to accurately reflect the content of the episode, whereas the posterior medial system seemed to distort the episode *as people were watching it* in a way that predicted how they would recount the episode later.
This suggests that even when different people experience the same (physical) thing, differences in how they process that experience and what they take away from it start to set in literally at the moment the experience is happening. 
Another aside: the brain networks we highlight fit very nicely with work by @maureenritchey and @CharanRanganath-- e.g., check out their fantastic review: https://tinyurl.com/1sqvql43
There's more cool stuff in the paper too; please check it out! Or if you'd rather read the abridged version, here's our "blog post" on the work: https://tinyurl.com/12zr36he
We think the framework we've developed can be used to study a very wide range of interesting problems regarding the dynamics of cognition. We've shared all of our materials to make this easy: https://tinyurl.com/1erd0z9g
Special shoutout to @kanileke, who generously shared all of her public data-- and when we told her what we were working on, she immediately sent us (and published) additional *unreleased* data she thought would be cool for us to look at. We're so grateful!
I'd also like to acknowledge the truly fantastic reviewer feedback we received. This was a very very long journey, but the peer review system actually worked here. Kudos to @NatureHumBehav and senior editor @marike_cogneuro for a job well done!
Finally, it was wonderful seeing @andyheusser and @paxt0n4 take this from a vague malformed idea into a solid story and polished final product. You've done the lab proud!
