AIPsy-Affect: Stimulus Battery for Emotion in Language Models
AIPsy-Affect, a clinical stimulus battery comprising 480 items, has been introduced by researchers to evaluate emotion detection in large language models without relying on emotional keywords. This tool addresses a significant issue in mechanistic interpretability research: when a probe activates on 'I am furious', it remains ambiguous whether the model recognizes anger or simply the term 'furious'. The battery features 192 narrative vignettes that evoke each of Plutchik's eight primary emotions without keywords, alongside 192 neutral control scenarios. It also includes splits for moderate intensity and discriminant validity, enabling researchers to differentiate between true emotion detection and keyword identification through various analytical methods such as linear probing and causal ablation.
Key facts
- AIPsy-Affect is a 480-item clinical stimulus battery.
- It removes the confound of emotion keywords at the stimulus level.
- Includes 192 keyword-free vignettes for each of Plutchik's eight primary emotions.
- Includes 192 matched neutral controls.
- Includes moderate-intensity and discriminant-validity splits.
- Designed for mechanistic interpretability of emotion in language models.
- Addresses linear probing, activation patching, SAE feature analysis, causal ablation, steering vector extraction.
- Released on arXiv with ID 2604.23719.
Entities
Institutions
- arXiv