AI game coding jumps 5x with strict-launch filter over data duplication
A simple 'does it launch?' test beats adding more training data for code generation.
A new paper, 'The Verifier is the Curriculum: Execution-Gated Self-Distillation for Cross-Family Game Generation,' tackles a core AI training problem: post-training a code generator against a learned judge can optimize for proxy features without real improvement. The authors propose a deterministic, ungameable filter: whether a generated Godot project launches cleanly under a headless engine ('strict-launch'). Using this gate, rejection-sampling self-distillation allows a 14B parameter Qwen3-14B model (fine-tuned with LoRA) to compound out-of-family generalization on GameCraft-Bench—a benchmark mapping natural language briefs to complete Godot projects.
Results are striking: clean generation on four unseen game families jumped from 8.8% to 42.2% per-candidate over three rounds, and best-of-K coverage went from 18/25 to perfect 25/25 (p<0.0001). The gain is not from added data alone—an exactly matched gold-duplication control regressed below the base model (5.6% vs 8.8%). A lenient BUILD check (passes 99.9% of generations) erased gains entirely. Headless execution grounding rose monotonically across rounds, confirming functional grounding. The lesson: the verifier is the curriculum—what it certifies is what the model learns.
- Strict-launch filter boosted clean game generation from 8.8% to 42.2% on unseen families (5x improvement).
- Best-of-K coverage reached perfect 25/25, while data duplication regressed to 5.6%.
- A lenient BUILD check (99.9% pass rate) erased all gains, proving verifier precision matters most.
Why It Matters
Demonstrates that a simple, task-specific verifier can outperform massive data scaling for code generation.