New theorem explains why different learning rates work in multi-agent AI training

Researchers Hédi Hadiji and Sarah Sachs have published a theoretical breakthrough in multi-agent learning. Their paper, “Stability and Convergence of Optimistic Exponential Weights with Asymmetric Step Sizes in Bimatrix Games,” tackles a fundamental problem: how do different learning rates (step sizes) affect the behavior of two AI agents playing repeated games? Previous work assumed both agents used the same step size, but real-world applications often require different rates (e.g., when one agent is faster or more constrained). The authors allowed η_x and η_y to differ and discovered that the key quantity is their product, not each rate alone. For zero-sum games, if that product lies below a certain threshold and the set of fixed points is finite, global last-iterate convergence is guaranteed—meaning both agents’ strategies will eventually stabilize at an equilibrium. This condition is practically relevant: it explains why practitioners often see convergence even when using different learning rates, as long as the product isn’t too large.

For general bimatrix games (not necessarily zero-sum), the team established an almost-tight threshold for asymptotic stability and instability. Strategies that satisfy this stability condition will resist small perturbations, while those that violate it will diverge. The results unify several known special cases (e.g., symmetric step sizes, two-strategy games) and provide explicit step-size bounds that can be used when training multi-agent reinforcement learning systems. The paper also includes experiments that validate the theory. While the primary contribution is theoretical, the implications are direct for anyone fine-tuning large-scale multi-agent simulations: the product of learning rates is the critical lever, not the individual rates. This could simplify hyperparameter tuning in competitive settings like poker, auctions, or automated negotiation systems. The paper is available on arXiv (2607.07517).