ToM Improvements in LLMs Don't Always Benefit Human-AI Interactions

A new study from arXiv challenges the assumption that improving Theory of Mind (ToM) in Large Language Models (LLMs) directly enhances human-AI interactions. The researchers propose an interactive evaluation paradigm that shifts from static, third-person benchmarks to first-person, dynamic, and open-ended assessments. They tested four ToM enhancement techniques across goal-oriented tasks (coding, math) and experience-oriented tasks (counseling), using four real-world datasets and a user study. Results show that improvements on static benchmarks do not consistently translate to better interactive performance. The findings highlight the need for evaluation methods that reflect the actual dynamics of human-AI communication.