New paper formalizes constrained correlated equilibria for multi-agent Markov games

A new arXiv paper by Tingting Ni, Anna Maddux, and Maryam Kamgarpour tackles a key problem in multi-agent decision-making under real-world constraints. Markov games with coupling constraints model situations where self-interested agents must coordinate their strategies because feasibility for one depends on the actions of others—think traffic systems with shared road capacity or electricity markets with a collective emissions cap. The authors introduce a formal definition of 'constrained correlated equilibrium' for such games, where a joint policy is considered stable if any agent's unilateral deviation either yields no profit or becomes physically infeasible due to the constraints.

Beyond the definition, the paper provides a novel existence proof for constrained correlated equilibria in Markov games with coupling constraints. This is a significant theoretical step because prior work on correlated equilibria largely assumed unconstrained settings. The proof leverages techniques from game theory and optimization to show that as long as agents face shared feasibility boundaries, such equilibria always exist. The practical implication: we can now design learning algorithms and solution methods for applications like autonomous vehicle coordination, smart grid management, and environmental regulation, where safety and budget limits are non-negotiable.