SceneFactory: GPU-Accelerated Driving Sim Hits 127x Faster Training

SceneFactory is a new GPU-vectorized platform for autonomous driving simulation that solves the long-standing trade-off between physical fidelity and parallel scalability. Built on NVIDIA Isaac Sim and Isaac Lab, it represents worlds and agents as batched tensors, running control, observations, rewards, and policy inference entirely as GPU tensor operations. It can convert Waymo Open Motion Dataset road topologies into simulation-ready USD worlds and populate them with multiple articulated PhysX vehicles, modeling precipitation and road-surface type as material friction coefficients. This enables concurrent simulation of hundreds of scenarios on a single GPU.

Performance-wise, SceneFactory achieves up to 127× higher throughput than a non-vectorized PhysX baseline on the same GPU, reaching 19,250 controlled-agent simulation steps per second at 256 worlds × 16 agents. Cross-simulator transfer tests reveal an asymmetric dynamics gap: physics-grounded RL policies transfer to a simplified kinematic bicycle model with 99.5% success, but reverse transfer drops to 47.3%. Under wet-road friction, friction-aware policies reduce mean peak DRAC from 58.7 to 27.8 m/s² without sacrificing goal reach. SceneFactory demonstrates that scalable autonomous-driving training need not discard articulated rigid-body dynamics or physically grounded road-condition variation, paving the way for more realistic, large-scale RL for self-driving cars.