HorizonForge: Driving Scene Editing with Any Trajectories and Any Vehicles

A research team from UC San Diego and NEC Labs has introduced HorizonForge, a groundbreaking AI framework for controllable driving scene generation accepted at CVPR 2026. The system addresses a critical bottleneck in autonomous driving simulation by enabling photorealistic editing of driving scenes with precise control over vehicle trajectories and object placement. Unlike previous approaches that struggled to balance photorealism with controllability, HorizonForge establishes a new paradigm that combines 3D scene representation with advanced video synthesis, achieving an impressive 83.4% user-preference gain over state-of-the-art methods.

The technical breakthrough lies in HorizonForge's dual representation approach, where scenes are reconstructed as editable 3D Gaussian Splats and Meshes, then rendered through a noise-aware video diffusion process that enforces spatial and temporal consistency. This allows users to modify vehicle trajectories, insert new objects via natural language commands, and generate diverse scene variations in a single feed-forward pass without computationally expensive per-trajectory optimization. The researchers also introduced HorizonSuite, a comprehensive benchmark for evaluating ego- and agent-level editing tasks. Experiments demonstrate that their Gaussian-Mesh representation delivers substantially higher fidelity than alternative 3D representations, while temporal priors from video diffusion prove essential for coherent synthesis across frames.