Coordination Matters: Evaluation of Cooperative Multi-Agent Reinforcement Learning

A new paper from Cardei, Landers, and Doryab challenges how we evaluate cooperative multi-agent reinforcement learning (MARL). Current benchmarks focus on aggregate metrics like return, success rate, or completion time—but the authors argue these hide crucial coordination dynamics. To expose them, they introduce STAT, a controlled testbed with commitment-constrained spatial task allocation that scales agents, tasks, and environment size while keeping observation access fixed.

Evaluating six representative value-based MARL methods (varying centralization), they find that similar return trends can mask vastly different coordination mechanisms: differences in redundant assignments, assignment diversity, and task-completion efficiency. The results show that under commitment constraints, performance is shaped not only by action-space size but also by assignment pressure, sparse decision opportunities, and redundant choices among interdependent agents. The paper calls for coordination-aware evaluation as a necessary supplement to return-based benchmarking to truly advance multi-agent AI systems.