Whisper-Based Persian Speech Emotion Recognition with PCA Dimensionality Reduction
A new study shared on arXiv (2608.05165) focuses on recognizing emotions in speech for Persian, a language that lacks abundant resources. The researchers propose a method that extracts frame-level embeddings using OpenAI's Whisper model. They apply Principal Component Analysis (PCA) to streamline the data, cutting down on the need for complex projection layers and minimizing the number of parameters that need training. These efficient representations are then merged using an attention-based pooling technique and classified with a simple prediction head. They also explore whether fine-tuning Whisper on a Persian ASR task boosts SER performance. Tests on the ShEMO dataset show that PCA effectively reduces dimensions while maintaining competitive results, addressing the challenges of limited labeled data in low-resource languages.
Key facts
- arXiv paper 2608.05165 focuses on Persian Speech Emotion Recognition (SER).
- Uses Whisper encoder for feature extraction.
- PCA reduces dimensionality of frame-level embeddings, removing learned projection layers.
- Attention-based pooling aggregates reduced representations.
- A lightweight prediction head classifies emotions.
- Fine-tuning Whisper on Persian ASR is tested for SER improvement.
- Experiments use ShEMO dataset with speaker-independent evaluation.
- PCA-based method reduces trainable parameters while maintaining performance.
Entities
Institutions
- OpenAI
- arXiv