Adaptive Nucleus Truncation Sampling boosts LLM reasoning by up to 5.2 points
New ANTS method adapts token selection dynamically, boosting long-form reasoning by 5+ points
Existing truncation sampling methods like top-p and min-p rely on fixed thresholds that fail to adapt as entropy, task difficulty, or generation length changes during long-form reasoning. Adaptive Nucleus Truncation Sampling (ANTS) solves this by dynamically adjusting the truncation width through an entropy-conditioned controller. It extends top-nσ sampling into a rollout-control mechanism, selecting standardized neighborhoods around the maximum logit before temperature scaling, and includes a no-truncation fallback arm to stabilize training when aggressive truncation would harm performance.
On a 33B-total / 4B-active sparse Mixture-of-Experts reasoning model, ANTS delivers consistent gains across long generation budgets: +1.9 points at 8K tokens, +3.8 at 16K, and +5.2 at 32K. The biggest improvements appear in instruction following (IFBench +10 at 32K) and mathematical reasoning (AIME 2025 +7). Code generation on Codeforces initially trails at 8K but reverses to substantial ELO gains at 16K and 32K, revealing an important budget-dependent interaction. These results position sampler design not just as a decoding hyperparameter but as a critical mechanism for stabilizing and scaling long-budget reasoning.
- ANTS adapts truncation width using an entropy-conditioned controller, overcoming fixed-threshold limitations of top-p and min-p sampling.
- On a 33B MoE model, average gains of +1.9, +3.8, and +5.2 points at 8K, 16K, and 32K generation budgets respectively.
- Largest improvements in instruction following (+10 IFBench at 32K) and math (+7 AIME 2025); code generation reverses a 8K deficit to strong gains at longer budgets.
Why It Matters
Sampler design becomes a key scaling factor for long-budget reasoning, not just a hyperparameter.