arXiv's SARE framework reveals LLMs think unevenly across reasoning steps

While chain-of-thought (CoT) prompting has become a standard way to elicit reasoning from large language models, most interpretability methods treat a full reasoning trace as one opaque sequence, collapsing depth into a single scalar. That leaves a fundamental question unanswered: how does the model actually allocate computational effort from one step to the next? A new arXiv preprint from researchers at UC San Diego, University of Illinois, and ByteDance tackles this head-on. They propose Step-Aware Reasoning Energy (SARE), a framework that quantifies effort at the granularity of individual CoT steps by computing Centered Kernel Alignment (CKA) between Gram matrices of token hidden states across adjacent transformer layers. Crucially, CKA captures inter-token relational structure without needing eigenvector alignment or cluster correspondence, making it robust and model-agnostic.

Applying SARE across six reasoning benchmarks (including GSM8K-style math and multi-step commonsense tasks) and three open-weight LLMs, the team found several striking patterns. First, reasoning energy is highly non-uniform across step types: the model visibly flares and dips in distinct phases, transitions that are completely invisible to trajectory-level metrics. Second, incorrect trajectories show systematically lower energy at critical reasoning junctions—suggesting that the model 'glosses over' the exact step where it goes wrong. Third, SARE-based features match or outperform output-based confidence baselines in most settings, meaning internal geometric dynamics encode predictive information beyond surface-level signals. This opens a practical path to failure detection, more interpretable agent monitoring, and step-level self-correction in LLM systems.