DDPG-based RL speeds autonomous vehicle path planning 10x faster

Path planning for autonomous vehicles in environments with threats (modeled as circular 'no-go' zones) is inherently nonlinear and nonconvex, making real-time decision-making challenging for traditional optimal control methods. To address this, researchers from the US Naval Research Laboratory and other institutions propose a reinforcement learning approach using Deep Deterministic Policy Gradient (DDPG). The agent is trained entirely in simulation, learning a direct mapping from its current state (position and heading) to a series of feasible actions that safely navigate to a destination while avoiding restricted zones. The reward function has three components: (a) an attractive field centered at the destination, (b) repulsive fields around each circular obstacle, and (c) a penalty on control energy (heading change magnitude) to favor straighter paths. Crucially, the trained agent can identify the largest possible set of starting positions from which a safe path to the destination exists, providing critical pre-mission planning intelligence.

Validation in simulation shows the DDPG-based agent produces effective paths while being significantly faster than a traditional pseudo-spectral optimal control method, making it a better fit for real-time applications like drone navigation or autonomous ground vehicles in contested environments. The work is published as arXiv:2606.07855 (14 pages, 12 figures). While the paper focuses on circular obstacles, the framework is extensible to more complex threats. The key advantage is speed: the neural network inference is orders of magnitude faster than repeatedly solving nonlinear optimization problems, enabling real-time replanning. This represents a practical step toward integrating deep RL into mission-critical robotics where computational time is a bottleneck.