ViPO: Massive Preference Dataset for Visual Generative Models
A new research paper introduces ViPO, a massive-scale preference dataset containing 1 million image pairs designed to improve visual generative models through preference optimization. The authors identify critical flaws in existing open-source datasets, including conflicting preference patterns where winners excel in some dimensions but underperform in others, low resolution, limited prompt diversity, and imbalanced distributions. To address noise in preference data, they propose Poly-DPO, an extension of the DPO objective that incorporates a polynomial term to dynamically adjust model confidence based on dataset characteristics. The work aims to enable effective scaling of preference optimization for visual generation, tackling both algorithmic and data bottlenecks.
Key facts
- ViPO dataset contains 1 million image pairs
- Existing datasets have conflicting preference patterns
- Poly-DPO extends DPO with a polynomial term
- Poly-DPO adjusts model confidence dynamically
- Datasets suffer from low resolution and limited prompt diversity
- The paper is from arXiv:2604.24953v2
- Preference optimization is crucial for visual generative models
- Naive optimization on noisy datasets fails to learn preferences
Entities
Institutions
- arXiv