Woodpecker Distillation lets weak AI models fix strong models' reasoning bugs

A new paper from Baidu-affiliated researchers (Dayu Wang, Jiaye Yang, and colleagues) introduces Woodpecker Distillation, a training method that lets weaker language models diagnose and repair reasoning errors in stronger models. The authors observe that large language models often fail on reasoning tasks even when they possess the underlying capability. These failures, they argue, stem from localized bugs in intermediate steps—not broad incompetence. Crucially, these bugs are repairable: inserting a short patch generated by a weak probe model after the same strong-model reasoning prefix can redirect the trajectory toward the correct solution.

However, directly fine-tuning the strong model on these weak patches or repaired trajectories doesn't reliably internalize the correction. The useful signal lies not in the intervention text itself, but in how it reshapes the future reasoning distribution. Woodpecker Distillation captures this by contrasting successful and unsuccessful weak-model patches at the same prefix, constructing a corrective teacher distribution from the induced future token predictions, and distilling that signal into the strong model. On mathematical reasoning benchmarks, the method consistently improves strong-model performance and beats direct imitation baselines, suggesting a scalable path to self-improvement without larger or stronger teachers.