Cognitive Model Reveals Why Users Struggle with AI Explanations

A recent study published on arXiv (2604.27354) explores the reasons behind the limited success of explainable AI (XAI) in enhancing user comprehension. The research concentrated on structured data and analyzed reasoning approaches for various XAI techniques—namely, none, feature importance, and feature attribution—within a forward simulation task. Strategies were derived from an initial formative study, followed by decision collection in a summative study. Through cognitive modeling, the researchers applied fundamental processes and discovered that their models aligned more closely with human decision-making than standard machine learning proxies, providing valuable insights into effective reasoning approaches.