Conditional Flow-VAE for Safety-Critical Traffic Scenario Generation

Safety-critical scenarios—like near-collisions or erratic driver behavior—are essential for developing robust autonomous vehicles (AVs), but they are extremely rare in natural driving logs. Manual scenario design doesn't scale, and adversarial optimization often yields unrealistic maneuvers. To address this, researchers from the University of Toronto (including Raquel Urtasun) propose Conditional Flow-VAE, a generative model that uses conditional latent flow matching to produce diverse, realistic edge cases. The key innovation is distribution matching: instead of explicitly programming dangerous situations, the model learns to morph nominal driving scenes into safety-critical rollouts by aligning their latent representations. The framework can leverage both simulated data and real-world driving logs, enabling it to generate scenarios that are both physically plausible and challenging.

The team evaluated Conditional Flow-VAE against baselines like VAEs and standard normalizing flows. Results show it consistently generates novel safety-critical scenarios (e.g., sudden cut-ins, pedestrian jaywalking, unexpected braking) that are more realistic than adversarial methods. The approach also scales efficiently—adding more data improves diversity without sacrificing realism. This is a significant step for AV safety, as it provides a data-driven way to stress-test perception and planning systems without needing millions of miles of real-world driving. Accepted at ICRA 2026, the work opens the door to using generative models for automated scenario synthesis in simulation and testing pipelines.