MAGIC: Multi-Step Advantage-Gated Causal Influence for Multi-agent Reinforcement Learning

A key challenge in multi-agent reinforcement learning (MARL) is designing reward signals that promote effective coordination. Existing approaches often struggle to capture the true long-term causal influence between agents, leading to suboptimal teamwork. To address this, researchers introduce MAGIC (Multi-step Advantage-Gated Interventional Causal MARL), a framework that extracts multi-step causal influences and converts them into intrinsic rewards. MAGIC employs causal intervention with conditional mutual information to quantify how one agent's actions affect another's over multiple time steps, then uses an advantage-based gating mechanism to prioritize exploration toward goal-aligned behaviors.

In experiments across standard benchmarks like MPE (Multi-Agent Particle Environments) and SMAC/SMACv2 (StarCraft Multi-Agent Challenge), MAGIC outperformed state-of-the-art methods by a significant margin, achieving at least a 10.1% improvement in the main evaluation metric. This breakthrough has practical implications for domains requiring complex team coordination, such as autonomous drone swarms, warehouse robotics, and traffic management systems, where understanding agent-to-agent causal dynamics is critical for efficient operation.