Communication-Aware Multi-Agent Reinforcement Learning for Decentralized Cooperative UAV Deployment

A team of researchers including Enguang Fan, Yifan Chen, Zihan Shan, Matthew Caesar, and Jae Kim has published a new paper on arXiv detailing a communication-aware multi-agent reinforcement learning (MARL) framework. The system is designed to solve a critical problem for practical drone swarm deployment: how to maintain effective coordination when each agent has only a partial view of the environment and communication between drones is intermittent or limited by distance. Their architecture uses a centralized training with decentralized execution (CTDE) paradigm. During training, a centralized critic has access to the global state to learn an effective shared policy. For deployment, each UAV executes this policy independently, relying only on its own local sensor observations and messages received from nearby neighbors within a communication graph.

The core technical innovation involves two attention modules. An agent-entity attention module encodes the local state of a drone and the states of nearby entities in the environment. A neighbor self-attention module aggregates messages from other UAVs within communication range. The team evaluated the framework primarily on a cooperative relay deployment task called 'DroneConnect,' where drones must position themselves to provide network coverage to ground nodes. With 5 UAVs and 10 nodes, their method achieved 74% coverage under restricted communication, remaining competitive with a computationally intensive mixed-integer linear programming (MILP) solution that serves as an offline upper bound. Notably, the learned policy generalized to unseen team sizes without requiring fine-tuning. In a secondary test on an adversarial 'DroneCombat' task, the same framework transferred without modification and improved win rates over baselines that did not use communication.