Study Questions Reliability of LLMs for Assessing User States

A recent study published on arXiv questions the belief that large language models (LLMs) can accurately evaluate user states within conversational and adaptive systems. Titled "Can We Trust AI-Inferred User States," the research empirically investigates the psychometric reliability of AI metrics by conducting assessments across three bimodal LLMs: GPT-4o audio, Gemini 2.0 Flash, and Gemini 2.5 Flash. The authors focused on both individual score reliability and overall reliability, revealing that metric reliability is not inherently present in interpretive contexts. The instability observed at the individual score level indicates potential challenges in real-time adaptations based on these metrics.