PathCal: Training-Free Decoding Cuts AI Reasoning Length, Preserves Accuracy

Large Reasoning Language Models (LRMs) generate long Chain-of-Thought (CoT) trajectories during inference, often using explicit reflection markers like 'wait', 'but', and 'alternatively' to signal hesitation, revision, or alternative paths. Previous work treated these markers as a single category, missing their distinct functional roles. In a new paper, researchers from multiple institutions propose PathCal, a training-free decoding controller that performs type-wise suppression and fixed-prefix intervention. They discovered that different marker classes affect accuracy and generation length differently, and that marker choices are most consequential before the model settles into a stable reasoning trajectory.

PathCal leverages these insights to estimate local competition between maintaining the current reasoning path and initiating a competing branch. At each decoding step, it analyzes the distribution over reflection markers and softly rebalances logits when evidence for a competing branch becomes excessive. This selective intervention reduces unnecessary verbosity while preserving or even improving accuracy. Experiments on six reasoning benchmarks show PathCal achieves a superior efficiency-performance trade-off, all without relying on external verifiers or additional sampling. The approach is particularly promising for deploying LLMs in cost-sensitive or latency-critical applications.