LLM Narrative Explanations Don't Improve Human Decision Accuracy
A significant human behavioral experiment examined if narrative explanations generated by LLMs enhance decision-making in classification tasks. Results indicated that the effectiveness of these narrative explanations did not significantly influence decision accuracy compared to relying solely on a basic AI prediction, aligning with earlier research on explainable AI focused on feature importance. Participants engaged in the study were tasked with making decisions using LLM explanations that varied in persuasiveness, yet no notable improvement in performance was detected.
Key facts
- Large-scale human behavioral experiment conducted
- Evaluated decision-making performance with LLM-generated narrative explanations
- Varying levels of persuasiveness tested
- No meaningful impact on decision accuracy found
- Results agree with typical explainable AI findings
- Study published on arXiv (2605.23867)
- LLMs can generate cogent narrative explanations
- Prior work found narrative explanations understandable and trustworthy
Entities
Institutions
- arXiv