New ICML study shows preferences don't predict multi-agent learning outcomes

In multi-agent AI systems, agents learn and adapt. A central question is: can we predict what they'll do just by knowing their preferences? A new paper by Omar Abbadi, Rida Laraki, and Panayotis Mertikopoulos, presented as an oral at ICML 2026, answers with a nuanced 'no.' The authors study no-regret learning dynamics—specifically Follow-the-Regularized-Leader (FTRL)—and compare their long-run outcomes to the game's preference graph, a combinatorial structure capturing which pure strategies players prefer over others.

They first prove a positive result: any dynamically stable set (the set of pure profiles that survive learning) must be 'preferentially stable'—closed under profitable deviations. But the converse fails spectacularly. Preferences fully characterize stability only in restricted subgames. The authors construct a three-player game where a preferentially stable set is dynamically unstable under FTRL, showing that ordinal preferences are insufficient for predicting behavior. To close this gap, they introduce 'resilience under aggregate deviations,' a tractable payoff-based condition that guarantees asymptotic stability for arbitrary spans of pure strategies. This gives game designers and AI engineers a practical criterion for verifying whether a learning system will converge to a desired outcome, without having to simulate every possible trajectory.