GOMA achieves optimal O(1/k²) convergence for min-max optimization

A team of researchers—Motahareh Sohrabi, Jianxin You, Simon Lacoste-Julien, Eduard Gorbunov, and Gauthier Gidel—has proposed a new family of first-order methods called GOMA (Generalized Optimistic Methods with Anchoring) for solving monotone variational inequalities in min-max optimization. Classical workhorses like the extragradient method require two gradient queries per iteration, a bottleneck that complicates analysis and limits use in online or stochastic settings. GOMA instead combines two-time-scale optimistic updates with an anchoring term inspired by Halpern iteration, enabling single-query iteration while preserving strong convergence guarantees.

In the deterministic setting, GOMA achieves the optimal accelerated last-iterate rate of O(1/k²) on the squared gradient norm for monotone Lipschitz operators. For stochastic settings with unbounded variance, a simplified single-call variant reaches O(1/√k) last-iterate convergence—the first such guarantee for stochastic monotone Lipschitz variational inequalities in the unconstrained setting, and crucially, without relying on variance reduction or growing batch sizes. This removes a major practical constraint: previous stochastic algorithms often needed increasingly large batches or complex variance-reduction schemes to handle unbounded noise.

The work directly impacts applications like GAN training, adversarial machine learning, and multi-agent reinforcement learning, where min-max optimization is foundational. GOMA's provably stable and accelerated convergence could translate to faster, more robust training for those models. By eliminating the two-query requirement and anchoring the iterates, it also offers a simpler path to stochastic and online deployment. The paper, posted on arXiv (2606.21528), provides theory and suggests practical promise. Future work may extend these methods to broader operator classes and real-world benchmarks, but the current results already mark a notable step forward in optimization theory.