HPA method rethinks priority scheduling for multi-agent Stackelberg games
Changing agent order shifts equilibrium–researchers propose adaptive scheduling for better coordination.
In multi-agent systems modeled as N-level Stackelberg games, current research typically takes the default decision order of agents as given. A new paper from Xiangyu Liu, Liang Zhang, Bo Jin, and Ziqi Wei challenges this assumption by formally proving that altering the order in which agents make decisions can shift the game's equilibrium point, unless special structural conditions hold. This insight reveals that the default order may lead to suboptimal outcomes, especially in dynamic environments where agent roles and priorities should adapt.
The team introduces Hierarchical Priority Adjustment (HPA), a two-level framework that treats the decision order itself as a variable to optimize. At the upper level, a policy continuously selects the best ordering based on the current game state. At the lower level, agents execute within a Spatio-Temporal Sequential Markov Game (STMG) under that order. To synchronize learning across these time scales, HPA uses a slow-fast update scheme where the upper policy's advantage function generates shared intrinsic rewards for all agents. Tested on high-precision multi-agent MuJoCo tasks, HPA consistently outperforms baseline algorithms and shows robust adaptation to changing environmental conditions, underscoring the critical role of decision-order optimization in hierarchical multi-agent settings.
- Formally shows that changing agent decision order in N-level Stackelberg games shifts equilibrium points unless special structural conditions hold.
- Proposes Hierarchical Priority Adjustment (HPA) method with an upper policy that dynamically selects optimal decision order and a lower-level Spatio-Temporal Sequential Markov Game.
- Outperforms benchmark algorithms on multi-agent MuJoCo high-precision control tasks, demonstrating robust adaptation to changing environments.
Why It Matters
Optimizing agent decision order could significantly improve coordination in autonomous systems like robotics, traffic management, and drone swarms.