AI 'Poker Face' Effect Revealed in Multimodal Emotion Recognition Study

A recent study featured on arXiv (2605.20200) investigated a multimodal emotion recognition system incorporated into a proactive Socially Interactive Agent (SIA) that utilizes generative AI. This technology evaluated emotional states in real-time through two methods: facial recognition using computer vision and semantic analysis of language. In an empirical study involving 20 participants engaged in spontaneous conversations, researchers discovered a notable gap between the automated visual signals and the users' true emotional states. Participants often displayed a 'poker face'—serious, focused expressions even when feeling positive emotions. In contrast, the linguistic analysis conducted by the generative AI was found to be more dependable by providing context to the spoken words.