Emotional Circuits in Large Vision-Language Models Decoded via Causal Framework

A new causal attribution framework based on steering vectors has been proposed by researchers to elucidate the emotional processing of Large Vision-Language Models (LVLMs). They developed a dedicated dataset to examine a three-phase mechanism known as 'Adapt-Aggregate-Execute.' The findings revealed a functional decoupling: emotional visual signals are combined in intermediate layers using sentiment-specific attention heads, which are subsequently transformed into narrative output in deeper layers via emotion-general pathways. This research fills a critical gap in comprehending how LVLMs convert visual inputs into emotional storytelling, utilizing visual counterfactuals and causal analysis.