Qwen2.5-0.5B retrofit with recurrent depth beats scratchpad, runs 7.6x faster

A new arXiv paper by researcher Mark Shapiro demonstrates that a dense pretrained language model can be retrofitted with recurrent depth to perform iterative reasoning in latent space. The approach splits Qwen2.5-0.5B-Instruct into a Prelude, a weight-tied Recurrent Block, and a Coda, using an identity-preserving one-loop path and a re-entry bridge on later loops. The retrofit installs at two budgets: a lightweight adapter with just 6M trained parameters over frozen base weights, and a full 180M-parameter block. With intermediate-step supervision, the model computes one task step per loop and persists even when only final answers are graded. The 6M-parameter adapter closely matched the full block overall (83.8% vs 84.0%), actually leading through depth 11 before trailing beyond.

Compared against a same-size scratchpad-trained model, the recurrent model won overall (84% vs 72%), retained far more capability beyond its training horizon (53% vs 2.5% accuracy past depth 10), and answered 7.6 times faster. It also extrapolates to roughly 1.5 times its supervised depth, holding 70% accuracy through depth 18. However, the paper exposes a critical limitation: attempting to learn the inverse rule in a second task caused catastrophic interference, with no continuation preserving the installed mechanism and general capability. The author notes that learned depth selection remains open. Overall, the results suggest that iterative transformers can perform deeper reasoning in latent space faster than comparable or larger models fine-tuned with explicit scratchpads, offering a promising efficiency path for production systems.