Researchers' Skill Training lifts LLM pass rates by 8.1pp

A new arXiv paper from Fei Ding and colleagues introduces Scaffold-Mediated Post-Training, a paradigm that breaks the conventional separation between LLM parameter training and inference-time procedural scaffolds. Instead of treating scaffolds (like chain-of-thought or tool-use workflows) as fixed external add-ons, the method organizes them into an evolvable graph that co-evolves with model parameters. Through a loop of discovery, distillation, and dynamic recompilation, the model and scaffold graph iteratively improve each other, allowing complex strategies to be gradually internalized into the model's weights.

Instantiated as 'Skill Training,' this approach was tested on FeatureBench, where automatically discovered skills boosted the passed rate by 8.1 percentage points. After progressive distillation, the model still achieved a 27.7% passed rate even when running without any external scaffold — a retention rate of 85.2% compared to the with-scaffold performance. This significantly outperforms standard supervised fine-tuning (SFT) on the same data, suggesting that co-evolving scaffolds with parameters enables models to acquire and internalize strategies more effectively. The work points toward a future where post-training goes beyond static datasets, treating inference-time scaffolding as a trainable component that ultimately makes models stronger even when the scaffold is removed.