CADENCE: Interpreting ECG Foundation Models with Sparse Autoencoders
A group of researchers has unveiled CADENCE, a new framework that utilizes a BatchTopK sparse autoencoder to deconstruct embeddings from an ECG foundation model into 8,192 understandable cardiac components. By analyzing over nine million ECG tokens from Layer-6 of the model, they achieved average AUROCs of 0.88 for clinical phenotypes and 0.90 for waveform morphology, outperforming traditional dense embeddings. These components align with various clinical concepts, such as arrhythmias, conduction issues, infarction, and specific waveform characteristics related to leads.
Key facts
- CADENCE uses a BatchTopK sparse autoencoder.
- It decomposes Layer-6 embeddings from over nine million ECG tokens.
- The framework produces 8,192 sparse cardiac atoms.
- Atoms align with clinical phenotypes and waveform morphology.
- Best atoms achieve mean AUROCs of 0.88 for clinical phenotypes.
- Best atoms achieve mean AUROCs of 0.90 for morphology.
- Dense dimensions achieve 0.78 and 0.83 respectively.
- Sparse atom probes match or outperform dense probes.
Entities
—