PROWL uses adversarial curriculum to boost world model reliability on rare events

Modern video world models excel at short-term visual realism but fail on rare, interaction-critical transitions—exactly the moments that matter most for downstream planning and policy. A new paper from researchers at Google DeepMind and UCL introduces PROWL (Prioritized Regret-Driven Optimization for World Model Learning) to solve this. The key idea: instead of waiting for failures to occur naturally, train a separate policy to deliberately expose the world model's weaknesses. That policy operates under a KL-constrained adversarial curriculum, staying close to the behavior distribution while generating trajectories where the diffusion-based world model makes high prediction errors. The world model is then continuously fine-tuned on these adversarially discovered trajectories, creating a stable training loop that converts rare failures into near-distribution signals without drifting into out-of-distribution exploitation.

To keep pressure on unresolved weaknesses as the model improves, PROWL introduces a Prioritized Adversarial Trajectory (PAT) buffer that re-ranks trajectories based on prediction error, action fidelity, and learning progress. This ensures training focuses on genuine blind spots rather than repeatedly revisiting solved cases. Evaluated in the MineRL framework on held-out out-of-distribution trajectories, PROWL significantly improved robustness over models trained on passive data alone. It also revealed reward-hacking behaviors when behavioral constraints were too weak, and demonstrated that effective adversarial world-model training critically depends on balancing exploratory failure discovery with explicit behavioral regularization. The results suggest that scalable world models benefit not just from larger datasets, but from selectively generating informative training data.