TRIDENT MARL framework cuts training violations by 95.5% with provable safety
A novel algorithm breaks the three-way coupling between safety, physics, and hybrid actions.
Safe coordination in networked cyber-physical systems (like drone swarms or autonomous intersections) requires learning algorithms to juggle three competing demands: hybrid discrete-continuous actions, hard safety constraints during training, and physics-governed dynamics. The researchers prove that naive combinations of existing methods create a 'directed cycle of biases' that amplifies errors. They formalize this as a three-way coupling lemma, showing why off-the-shelf modules fail.
TRIDENT breaks this coupling with three co-designed innovations: a Richardson-Romberg gradient correction that slashes Gumbel-Softmax discretization bias from O(τ) to O(τ²); a Lyapunov-constrained sequential trust-region update that guarantees safety at every iteration (not just asymptotically); and a physics-informed residual critic that decomposes value rather than reward for better credit assignment. The framework achieves an O~(1/√K) convergence rate to a constrained Nash equilibrium with an O(√K) cumulative-violation bound. In benchmark tests—multi-UAV mobile edge computing, autonomous intersection management, and a hybrid StarCraft Multi-Agent Challenge variant—TRIDENT reduces training-time violations by 95.5% over MADDPG and 76.3% over MACPO, while boosting reward 13.5% above the strongest unconstrained baseline.
- Richardson-Romberg gradient correction reduces Gumbel-Softmax discretization bias from O(τ) to O(τ²)
- Lyapunov-constrained sequential trust-region update ensures safety constraints are met at every training iteration
- 95.5% fewer safety violations than MADDPG and 76.3% fewer than MACPO across multi-UAV, intersection, and SMAC scenarios
Why It Matters
Provable safety guarantees for multi-agent RL bring autonomous drones and self-driving intersections closer to real-world deployment.