New DR-MARL framework cuts worst-case traffic queues by 74%

Standard multi-agent reinforcement learning (MARL) for traffic signal control optimizes for average conditions, leaving systems vulnerable to spatial-temporal demand shifts and catastrophic congestion. To address this, researchers introduce an algorithm-agnostic Distributionally Robust (DR) MARL framework with an adaptive Contextual-Bandit Worst-Case Estimator (CB-WCE). The CB-WCE operates on a slower timescale, co-evolving with controllers to generate adversarial demand mixtures during training. This steers the learning process toward robustness against bottleneck scenarios without modifying underlying MARL architectures.

Evaluated across value-based, actor-critic, and policy-gradient methods on both a synthetic 5x5 grid and a heterogeneous Monaco City network, the framework prevents unbounded queue growth. For PPO in the Monaco environment, robust retraining reduced worst-case queue length by 74.39% and improved average-case network-wide queue by 75.45%. The retrained policies also exhibit strong zero-shot generalization to unseen traffic distributions, highlighting scalability for resilient real-world urban deployment.