TUX: A New Metric for Human-AI Tacit Understanding
A new metric called the Tacit Understanding Index (TUX) has been unveiled by researchers to assess the tacit understanding between humans and AI agents. Drawing inspiration from the social game Wavelength, the study created a task where participants and LLM agents position concepts along subjective spectra. TUX measures the similarity between the judgments of humans and agents. The research included 241 human participants and 200 profile-conditioned LLM agents from four different models. Findings reveal that the human-agent pairs closest in trait space demonstrate significantly elevated TUX scores, indicating that tacit alignment is influenced by individual characteristics. This research fills a gap in assessing human-AI alignment beyond straightforward task success, emphasizing implicit coordination without defined objectives or feedback.
Key facts
- TUX measures tacit understanding between humans and AI agents.
- Task inspired by the social party game Wavelength.
- 241 human participants and 200 LLM agents tested.
- Four different LLM models were evaluated.
- Nearest human-agent pairs in trait space had higher TUX.
- Tacit alignment is structured by person-level traits.
- Study operationalizes tacit understanding as pairwise similarity.
- Published on arXiv under ID 2605.30930.
Entities
Institutions
- arXiv