SARE: A Geometric Framework to Measure Step-Wise Reasoning Energy in LLMs
A recent preprint on arXiv (2607.28674) presents Step-Aware Reasoning Energy (SARE), a geometric model designed to assess computational effort at the individual reasoning step level in large language models (LLMs). This approach leverages Centered Kernel Alignment (CKA) to analyze Gram matrices of token hidden states from adjacent transformer layers, effectively capturing the relational structure between tokens without the need for eigenvector alignment or cluster matching. Additionally, SARE interprets CoT trajectories as movements through latent semantic states, providing context for energy within the reasoning process. Evaluating SARE on six reasoning benchmarks and three open-weight LLMs reveals significant variability in reasoning energy across different step types. This framework enhances interpretability, addressing the opacity of step-wise effort compared to traditional output-level or trajectory-level methods.
Key facts
- SARE is a geometric framework for quantifying reasoning effort per CoT step.
- It uses Centered Kernel Alignment (CKA) between Gram matrices of token hidden states across adjacent transformer layers.
- SARE does not require eigenvector alignment or cluster correspondence.
- It models CoT trajectories as transitions among latent semantic states.
- The study covers six reasoning benchmarks and three open-weight LLMs.
- Findings show reasoning energy is highly non-uniform across step types.
- The preprint is available on arXiv with ID 2607.28674.
- The method aims to make step-wise effort in LLM reasoning more transparent.
Entities
Institutions
- arXiv