EEG Predictions Unstable Under Preprocessing Changes, Study Finds
A recent arXiv study indicates that EEG decoding deep learning models are significantly affected by the choices made during preprocessing. Researchers conceptualized preprocessing as a counterfactual intervention space and discovered that altering the preprocessing pipeline can lead to a reversal in up to 42% of trial-level predictions across six datasets from four different paradigms. This inconsistency is overlooked by conventional uncertainty methods. The research presents three innovative tools: a Walsh-Hadamard decomposition of the 2^7 pipeline space demonstrating near-additive sensitivity, a Preprocessing Uncertainty metric, and a technique to enhance stability. These results expose a major issue in EEG-based brain-computer interfaces and clinical neuroscience, where models are often developed using a single, unreported preprocessing approach.
Key facts
- Up to 42% of trial-level EEG predictions flip with different preprocessing.
- Study analyzed six datasets across four paradigms.
- Preprocessing choices formalized as counterfactual intervention space.
- Walsh-Hadamard decomposition used for 2^7 pipeline space.
- Standard uncertainty methods do not quantify preprocessing variability.
- Three tools introduced: decomposition, uncertainty metric, reduction method.
- Deep learning models in EEG typically use single unreported pipeline.
- Findings impact brain-computer interfaces and clinical neuroscience.
Entities
Institutions
- arXiv