MUPA2E: Unified Multimodal Framework for Emotion Assessment from Facial Video and EEG

There’s this new study that introduces a framework called MUPA2E, which stands for Multimodal Unified Perception with Asymmetric Attention for Emotion Assessment. It’s built to analyze both facial videos and EEG signals by using a unique asymmetric-attention approach. The goal is to improve how we automatically assess emotions by combining neural and behavioral data without needing separate processes for each type. In this method, facial videos are encoded with axis-folded frame tokens, while EEG can be used either as raw data or processed into the spatial domain. Tested on the DMER dataset, the best results came from merging video and EEG at a stride of 30. You can find the study on arXiv with the identifier 2608.15999.