BIRD method slashes chain-of-thought length by 64% while boosting accuracy

Large reasoning models typically generate verbose chain-of-thought (CoT) traces filled with redundant steps, self-verification loops, and detours. Existing on-policy self-distillation methods try to compress these traces by training a student model on concise copies of its own rollouts, but suffer from an initialization bottleneck: supervision is applied to noisy or off-track prefixes, limiting learning. To address this, researchers from (likely multiple institutions) introduce BIRD (Bootstrapped Iterative Self-Reasoning Distillation), a two-stage approach that first warms up the model by generating concise, answer-correct solutions under a brevity instruction and fine-tuning them with a prompt-switch technique. This turns instruction-induced conciseness into a default behavior. Then, on-policy reverse-KL distillation with a concise self-teacher is applied on cleaner prefixes, yielding much more effective compression.

BIRD achieves state-of-the-art results across the Qwen3 model family. On Qwen3-8B, MATH-500 accuracy jumps from 86.2% to 92.0% while average response length plummets from 3,099 to 1,115 tokens — a 64% reduction. Similar gains are seen on AIME benchmarks. The work demonstrates that improving the support of prefixes before on-policy training is a central factor in efficient reasoning distillation. For professionals deploying LLMs at scale, BIRD offers a practical way to drastically cut inference costs while actually improving accuracy, making it a significant advance in compressed reasoning.