HALO: Hybrid adaptive reasoning boosts LLMs with less compute
Selective token refinement beats full-sequence methods on MMLU and GPQA.
Micah Zhang's new paper presents HALO (Hybrid Adaptive Latent Reasoning), a technique that enhances a frozen pretrained language model by adding a small amount of adaptive extra computation. Traditional fixed refinement approaches either apply a single weak refinement step or force two full-sequence passes everywhere, the latter wasting compute. HALO combines a coarse refinement stage with a selective second-stage refinement that uses token scoring and monotonic token halting to decide which tokens truly need further processing. This hybrid strategy eliminates wasteful compute on trivial tokens while still providing deeper reasoning where it matters.
On public benchmarks built from MMLU-Pro and GPQA-Diamond, HALO achieves the best overall average among the paper-facing methods, outperforming both the frozen backbone and fixed-1/fixed-2 baselines. Internal analysis shows HALO reaches nearly the same token-accuracy level as fixed-2 while using fewer average applied refine steps than fixed-1 and far fewer than fixed-2. The key insight is that better allocation of refinement—not simply more refinement—drives performance gains. HALO demonstrates that intelligent selective compute can match or surpass full-sequence methods at lower cost, a promising direction for efficient LLM inference.
- HALO uses token scoring + monotonic halting to selectively apply second-stage refinement only on high-value tokens.
- Outperforms fixed-1 and fixed-2 baselines on MMLU-Pro and GPQA-Diamond benchmarks.
- Achieves fixed-2-level accuracy while using fewer refine steps than fixed-1 and significantly less controller compute.
Why It Matters
Smarter token refinement could slash inference costs while maintaining or improving reasoning quality.