ReflectDrive-2: Reinforcement-Learning-Aligned Self-Editing for Discrete Diffusion Driving

ReflectDrive-2, developed by a team of researchers including Huimin Wang and colleagues, is a new autonomous driving planner that operates on discrete trajectory tokens using a masked discrete diffusion model. Unlike traditional approaches that require separate refinement networks for error correction, ReflectDrive-2's AutoEdit mechanism allows the same model to rewrite selected tokens directly in the discrete token space. This in-place revision capability is trained in two stages: first, structure-aware perturbations along longitudinal and lateral directions are applied to expert trajectories, and the model learns to recover the original; second, the full decision-draft-reflect pipeline is fine-tuned with reinforcement learning (RL), assigning terminal driving reward to the final post-edit trajectory.

The results on the NAVSIM benchmark are striking. With camera-only input, ReflectDrive-2 achieves a PDMS (probabilistic driving metric score) of 91.0, and in a best-of-6 oracle setting it reaches 94.8 PDMS. Critically, the RL fine-tuning step is essential: under supervised pre-training alone, inference-time AutoEdit improves PDMS by only 0.3 points, but full-rollout RL increases that gain to 1.9 points. The system also co-designs an efficient reflective decoding stack that combines shared-prefix KV reuse, Alternating Step Decode, and fused on-device unmasking, enabling an average latency of just 31.8 ms on NVIDIA Thor hardware. This work demonstrates how RL-aligned self-editing can make discrete diffusion planners both accurate and fast for real-world driving.