New MARL method compensates for delays with learned filtering

Multi-agent reinforcement learning (MARL) systems deployed in the real world often suffer from stale observations, stochastic communication delays, and intermittent packet loss. Policies trained under idealized synchronous conditions degrade severely because they act on outdated feedback. In a new arXiv paper, researchers Maxim Mednikov and Oren Gal propose a modular execution-stage state-estimation layer called Decoupled Delay Compensation. The framework combines a learned Gated transition model with a recursive Kalman filtering layer to estimate instantaneous states from asynchronous measurements. Crucially, the estimator serves as a plug-in for pre-trained policies, requiring no modifications to the original MARL training algorithm, architecture, or reward structure. This makes it easy to retrofit existing systems for robustness against real-world communication imperfections.

The approach was evaluated across diverse multi-agent and continuous-control benchmarks. Results consistently show that the proposed layer enhances robustness to communication latency and message loss. The most significant performance gains occur in coordination-intensive and dynamically unstable tasks where temporal consistency is critical for control. For example, tasks such as multi-agent cooperative navigation and formation control saw substantial improvements in reward stability and task completion rates under high delay or packet loss scenarios. By enabling pre-trained agents to maintain performance under realistic network conditions, Decoupled Delay Compensation bridges the gap between simulated training and practical deployment.