INFUSER self-evolution boosts Qwen3 reasoning by 20%+ on Olympiad

Self-evolution promises scalable reasoning improvements for language models, but existing methods rely on curated data or reward difficulty heuristics that don't guarantee solver improvement. INFUSER, introduced by a team including Siyu Chen, Miao Lu, and others from Stanford, Princeton, and other institutions, solves this with a co-training framework: a Generator creates questions and golden answers from an unstructured document pool, while a Solver trains on correctness rewards against those answers. The key innovation is an optimizer-aware influence score that rewards the Generator only for questions that actually improve the Solver's target distribution performance. A new optimizer, DuGRPO (dual-normalized GRPO), handles the noisy influence score, turning the document pool into an adaptive curriculum that favors useful questions over merely hard ones.

On Qwen3-8B-Base, INFUSER beats strong self-evolution baselines with over 20% relative improvement on both Olympiad and SuperGPQA benchmarks. Remarkably, an 8B INFUSER co-evolving generator outperforms a frozen 32B thinking generator on math and coding tasks. Ablation studies confirm each design choice is necessary. Extensions to instruction-finetuned anchors and rule-verifiable RLVR data further demonstrate flexibility. The framework effectively lets models bootstrap their own reasoning improvement without human-curated datasets, offering a path to affordable, scalable AI reasoning enhancement.