Read on Twitter
We wrote a review/tutorial on methodological issues when analyzing human sleep EEG. Basically a "what I wish I'd known when I started doing this stuff" kind of thing. Now in Sleep Medicine Reviews (open access). Thread https://doi.org/10.1016/j.smrv.2020.101353
Sleep EEG analysis often involves extensive signal processing with an enormous number of analysis choices (and few universally accepted guidelines). Over time, I've come to "appreciate" that diff analysis strategies don't always converge, even when applied to the same dataset.
Sometimes this is due to sig process errors. Unfortunately, many doing sleep EEG (including me starting out) have relatively little sig process background, which can lead to both fundamental errors and the inability to recognize them.
But even when implemented correctly, two perfectly reasonable analysis approaches sometimes just yield diff results (and diff interpretations). And you'd never realize unless you compare approaches in the first place.
We discuss several issues, such as estimating PSD/power, reference options, extracting phase and amplitude info, surrogate construction, and how different choices in these realms affect downstream metrics like topographies, cross-freq coupling, and interregional phase synchrony.
Some specific insights: power spectra and topographies can look substantially diff for diff ways of calculating power. For example, "raw" and "relative" PSD (here: normalized to total PSD) can lead to different ideas about where spectral components are expressed.
Reference choice has dramatic effects on e.g. slow oscillation-spindle coupling, both in terms of where such dynamics are found, and the phase at which spindles peak (figure: phase). Similar story for phase synchrony.
For analyses using Hilbert or wavelets there are endless ways to get your phase or amplitude estimates wrong, at times wholly invalidating outcomes (def fell victim to some of these myself)
Surrogates are often used to determine whether some dynamic (like SO-spindle coupling) exists beyond chance levels. Again, diff shuffling approaches can lead to widely diff conclusions (way more than I imagined)
More broadly, without knowledge (theory and hands-on) of fundamental concepts (e.g., trigonometry, complex numbers, fft, convolution), it's difficult/impossible to verify correct implementation of more advanced metrics (e.g., cross-freq coupling).
We also consider how flexibility in analysis options may invite (unintended?) post hoc arguments in favor of the pipeline yielding the most interesting (i.e. significant) outcome. So easy to get false positives (consistent with non-replications of several high-profile papers)
At the same time, strategically varying analysis choices can instill confidence that the pipeline behaves as intended, and that results are robust to these choices. So a balancing act.
Most of what we say has been said before and is not specific to human sleep EEG, but we figured it might be helpful to repeat these messages for a sleep audience. (And if not, at least now I have it written down somewhere for future me.)
Code (Matlab, "for which I apologize") plus data to reproduce all figures and then some can be found here: http://doi.org/10.5281/zenodo.3712075
Final thought: while links between human sleep phys and cognition/disease are still intriguing, I've become much more skeptical of my/others' findings. Generally count new findings as a "maybe" until they've been replicated (which we need to do much more). Peace out.
