AI Study on Human Reliance: LLM Explanations Fall Short in Visual Reasoning
A new study posted on arXiv (2604.03237) investigates how various AI support formats affect human-AI team performance. Researchers ran three controlled experiments using RAVEN matrices for visual reasoning and LSAT problems for logical deduction. They observed that AI predictions and explanations impact objective accuracy and subjective confidence in distinct ways. In visual reasoning, explanations from large language models did not improve accuracy over the predicted answer alone; none of the tested support formats significantly outperformed prediction-only support. Probability outputs yielded the best descriptive accuracy and error recovery, while a selective automation policy served as a higher-performing reference benchmark. The findings point to task-specific strategies for calibrating human reliance on AI.
Key facts
- Paper available on arXiv with ID 2604.03237.
- Examines human decision-making with AI support.
- Three controlled human-subject studies conducted.
- Uses RAVEN matrices for visual reasoning and LSAT problems for logical reasoning.
- Large language model explanations do not increase accuracy compared to prediction alone in visual tasks.
- None of the support formats beat prediction-only support to a statistically significant degree.
- Probability outputs perform best in descriptive accuracy and error recovery.
- A selective automation policy is a reference benchmark with higher performance.
Entities
—