Stakeholder Grounding Exercise Reveals Embedding-Human Misalignment
A new study from arXiv (2605.27168) introduces the Stakeholder Grounding Exercise, a method to align text embeddings with expert human associations. In a primary case study on Danish policy issues, neural text embeddings showed a 19-26 percentage point gap in reliability compared to human experts, with misalignment propagating to downstream clustering performance (Spearman ρ=0.9). A secondary study on US Federal AI use cases replicated the gap (16pp) in English using a digital protocol with a different expert community, demonstrating the gap is not an artifact of language or setting. The method aims to make expert associations explicit and ground embedding results in human understanding for valid analyses of large text corpora.
Key facts
- Study from arXiv:2605.27168
- Introduces Stakeholder Grounding Exercise
- Primary case study on Danish policy issues
- Neural text embeddings 19-26 pp less reliable than human experts
- Misalignment propagates to clustering (Spearman ρ=0.9)
- Secondary study on US Federal AI use cases
- Replicated gap of 16pp in English
- Digital protocol used with different expert community
Entities
Institutions
- arXiv
Locations
- Denmark
- United States