DEGS: Training-free LLM reasoning matches RL with entropy signals

A new paper from arXiv proposes Depth-Entropy Guided Sampling (DEGS), a training-free test-time method that exploits the internal forward-pass dynamics of large language models to improve reasoning. The key insight is that stronger reasoners—including RL-post-trained models—exhibit a distinctive 'late collapse' in layer-wise entropy: token probability uncertainty stays high until deeper layers before converging. DEGS defines a per-sequence collapse depth and combines it with sequence likelihood in an MCMC power-sampling framework (DEGS-MCMC). Across three open-weight models and four benchmarks, this near-chance per-candidate signal compounds over sampling trajectories into state-of-the-art training-free accuracy, with gains largest on out-of-domain and harder splits where likelihood alone fails. The method trails an in-house GRPO reference on math splits but surpasses it on GPQA for all three models, requiring only single-digit-percent wall-clock overhead and no training, reward model, or labeled data.

DEGS represents a shift away from expensive reinforcement learning for reasoning, making high-quality reasoning accessible to anyone with a base model and inference compute. By exploiting a previously overlooked signal in the transformer's internal activations, the method achieves RL-level gains on certain tasks without any data curation or training pipeline. This opens the door for rapid, on-the-fly improvements that can be applied to any open-weight model, potentially democratizing advanced reasoning capabilities. The paper is available on arXiv under reference 2607.09693.