Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planning
New autonomous driving system boosts safety by 30.6% by aligning high-level decisions with low-level planning.
A research team led by Yuehao Song and 10 other authors has introduced Senna-2, a novel autonomous driving system designed to solve a critical problem in AI-powered vehicles: the misalignment between high-level reasoning and low-level control. Current systems often use Vision-Language Models (VLMs) for semantic understanding (e.g., "merge left") and separate end-to-end (E2E) policies for trajectory planning, but these components can work at cross-purposes, leading to unsafe or inefficient maneuvers. Senna-2 explicitly bridges this gap through a consistency-oriented, three-stage training paradigm that ensures the VLM's decisions directly guide the E2E policy's actions.
The first stage involves standard driving pre-training to establish basic capabilities. The second stage performs open-loop alignment, while the third and most crucial stage employs closed-loop alignment via bottom-up Hierarchical Reinforcement Learning (HRL) within realistic 3D Gaussian Splatting (3DGS) simulation environments. This HRL stage reinforces safety and efficiency by allowing the system to learn from consequences. A key technical innovation is the 'decision adapter,' which transmits VLM decisions to the E2E policy as implicit embeddings, creating a more cohesive control flow.
Extensive experimental results demonstrate significant improvements. Senna-2 achieves a 19.3% higher F1 score for dual-system consistency, meaning its high-level commands and low-level actions match more reliably. It also shows a 5.7% reduction in Final Displacement Error (FDE) in open-loop tests and, most importantly, a 30.6% reduction in the Accident-Free Collision Rate (AF-CR) in closed-loop simulations. This indicates a substantially safer and more decision-following autonomous agent. The work represents a meaningful step toward more trustworthy and predictable self-driving systems by ensuring their 'brain' and 'body' work in harmony.
- Achieves 30.6% reduction in collision rate (AF-CR) through closed-loop hierarchical reinforcement learning
- Boosts alignment between high-level decisions and low-level control by 19.3% (F1 score)
- Uses a three-stage training paradigm including simulation in 3DGS environments for safety reinforcement
Why It Matters
This directly addresses a core safety flaw in autonomous driving AI, making self-driving cars more predictable and reliable by ensuring their reasoning matches their actions.