PrefMoE Framework Addresses Group Preference Collapse in MLLMs
A study published on arXiv (2607.22603) presents PrefMoE, a novel framework designed to tackle group preference collapse in personalized multimodal large language models (MLLMs). This phenomenon occurs when MLLMs for multiple users become unresponsive to unique preferences, instead gravitating towards prevalent choices due to diminished preference signals and inconsistent preference application during generation. Current approaches focus on profile-level data, neglecting the variety of user preferences. PrefMoE distinguishes stable profile data from preference-related elements, breaking down preferences into common prototypes and individual residuals. It maintains these individualized residuals through imbalance-aware learning, counterfactual pseudo-user augmentation, and residual decorrelation, while directing profile and preference factors along distinct LoRA adaptation paths. Experiments across various MLLM frameworks demonstrate significant improvements.
Key facts
- Paper on arXiv: 2607.22603
- Title: Group Preference Collapse in Personalized Multimodal Large Language Models
- Identifies group preference collapse in multi-user personalized MLLMs
- Proposes PrefMoE framework
- PrefMoE separates profile information from preference representations
- Decomposes preferences into shared prototypes and personalized residuals
- Uses imbalance-aware learning, counterfactual pseudo-user augmentation, residual decorrelation
- Routes profile and preference through separate LoRA adaptation paths
- Experiments conducted across multiple MLLM backbones
Entities
Institutions
- arXiv