CLEAR: Closed-loop RL sets new state-of-the-art in autonomous driving
Scalable reinforcement learning closes the loop for end-to-end self-driving cars.
End-to-end autonomous driving (E2E-AD) aims to map raw sensor data directly to driving actions. Recent Vision-Language-Action (VLA) models combine perception, language, and action prediction, but they typically rely on imitation learning from logged expert trajectories. This creates a distribution shift during closed-loop inference, leading to suboptimal performance. To close this gap, the authors introduce CLEAR, which uses reinforcement learning (RL) to train a residual waypoint policy on top of a pretrained VLA policy, preserving its knowledge while learning from online experience.
Scaling RL for vision-based policies usually requires massive parallel simulation environments, but resource contention between simulators and learners limits throughput. CLEAR solves this with a heterogeneous pipeline—separating simulation and training onto distinct compute groups—enabling many more parallel environments without instability. Using a simple reward function, CLEAR significantly outperforms prior methods, setting new state-of-the-art results on the CARLA Longest6 v2 and Bench2Drive benchmarks. This work demonstrates that closed-loop RL can be practically scaled for autonomous driving, closing the loop on real-world performance.
- CLEAR learns a residual waypoint policy on top of pretrained VLA models to combine imitation and reinforcement learning.
- A heterogeneous pipeline decouples simulators and learners to scale to many parallel environments without resource contention.
- Achieves state-of-the-art on CARLA Longest6 v2 and Bench2Drive benchmarks using a simple reward function.
Why It Matters
Shows closed-loop RL is viable for autonomous driving, potentially safer and more robust real-world performance.