AI Diversity Measurement Called 'Under-Specified' in New arXiv Paper
A new arXiv paper (2608.17731) argues that diversity in AI-generated content cannot be reduced to a single number. Scalar scores can mislead. Existing methods embed samples, compute pairwise distances, and aggregate them into metrics, often yielding contradictory rankings. The authors review representative metrics and critique them via axiomatic and empirical analyses. No scalar metric satisfies all desirable properties, and high-dimensional spaces may cause concentration effects. The study suggests diversity profiles as an alternative.
Key facts
- Paper arXiv:2608.17731 addresses diversity evaluation for generative AI.
- Current approaches embed generated samples and compute pairwise distances.
- Scalar summaries can yield contradictory rankings of the same sample sets.
- Diversity evaluation is intrinsically under-specified as a single number.
- The paper includes an axiomatic analysis of diversity metrics.
- No representative scalar metric satisfies all desirable properties simultaneously.
- Empirical analysis indicates high-dimensional representation spaces induce concentration.
- The paper proposes diversity profiles as an alternative to scalar scores.
Entities
Institutions
- arXiv