Research Reveals Self-Reading Patterns in LLMs' Quantitative Reasoning Traces

A recent study investigates how large language models analyze their reasoning processes prior to formulating answers, particularly in quantitative reasoning tasks. The researchers scrutinized the attention dynamics between answer tokens and reasoning traces, uncovering a unique self-reading pattern linked to correct answers. This pattern is characterized by a forward movement of focus along the reasoning trace and sustained attention on significant semantic anchors. Conversely, incorrect answers exhibit scattered and inconsistent attention. The team views this as a sign of internal confidence during answer interpretation, where models adhere to plausible solution paths and incorporate crucial evidence. They introduce a novel, training-free method called Self-Reading Quality (SRQ) scores, which merges geometric and semantic metrics. This research, identified as arXiv 2604.19149v1, fills a critical gap in understanding how answer tokens engage with reasoning to yield dependable results, extending beyond previous activation steering studies that mainly concentrated on the reasoning traces themselves.