A Controllability Perspective on Steering Follow-the-Regularized-Leader Learners in Games

A team of researchers from the University of Toronto and the University of British Columbia has published a groundbreaking paper titled 'A Controllability Perspective on Steering Follow-the-Regularized-Leader Learners in Games' on arXiv. The work examines a critical scenario in multi-agent systems: what happens when one strategic agent (the controller) wants to influence other agents who are using popular Follow-the-Regularized-Leader (FTRL) algorithms—common in game theory and reinforcement learning. The researchers treat the learners' dynamics as a nonlinear control system and ask a fundamental question: can the controller steer these automated learners to a desired outcome without changing the game's rules or payoffs?

For the two-player case, the team provides a precise mathematical criterion for controllability, requiring both a 'fully mixed neutralizing controller strategy' and a specific rank condition on the payoff map. They extend this analysis to multi-learner interactions with two sufficient conditions, one based on uniform neutralization and another combining periodic-drift with Lie-algebra rank conditions. The paper illustrates these theoretical results with canonical examples including Rock-Paper-Scissors and a construction related to Brockett's integrator, demonstrating practical applications of their framework.

The implications are significant for understanding strategic manipulation in automated environments. As AI agents increasingly use FTRL and similar algorithms in everything from financial markets to autonomous vehicles, this research provides tools to analyze when such systems might be vulnerable to strategic influence. The work bridges control theory and game theory, offering new perspectives on security, robustness, and design of multi-agent AI systems where not all participants may have aligned objectives.