LLMs Hallucinate on Structured Knowledge Due to Internal Dynamics

A recent investigation published on arXiv (2605.26362) explores the reasons behind hallucinations in large language models (LLMs) when processing linearized structured information such as graphs and tables. The findings reveal that these hallucinations arise from systematic internal dynamics instead of mere random fluctuations. Researchers observed that attention tends to concentrate on shortcut-like structural indicators rather than evenly distributing across the entire context. Moreover, the feed-forward representations do not effectively anchor the information provided, leading the model to rely on its parametric memory. The study highlights a consistent link between hallucinations and semantic grounding failures in feed-forward layers, while attention distribution varies depending on the task. This analysis enhances the understanding of hallucinations in LLMs and suggests potential mitigation strategies.