Researchers derive Hamiltonian probability framework for Muon optimizer

Researchers from (presumably) an academic institution have provided a rigorous theoretical foundation for the Muon optimizer, a recent optimizer that has shown promising results in large-scale machine learning. The paper, titled "Move on Muon: A Hamiltonian probability gradient flow perspective of Muon optimizer," introduces a regularized version of Muon that smooths the orthogonalization step. This regularization is shown to be the gradient of a smooth Fenchel-dual smoothing of the nuclear norm, identifying the Muon update as a mirror descent step where momentum acts as the dual coordinate. Using this perspective, the authors lift the optimizer from a single matrix parameter to finite-particle probability objectives of the form J(ρ)=R(∫ F dρ), which models mean-field descriptions of neural network training. They derive the inertial continuous-time limit under specific scaling of step size and momentum.

The resulting dynamics is characterized as a damped Hamiltonian probability system, with the kinetic energy induced by the regularized Muon mirror potential. The authors prove an exact Hamiltonian dissipation identity, showing that the Hamiltonian energy decreases monotonically. While the target objective itself need not be monotone, they establish continuous and discrete-time exponential convergence rates for the objective gap under assumptions of gradient dominance, bounded momentum, and curvature/alignment. Additionally, they study well-posedness of the mean-field limit equation and provide propagation of chaos guarantees. The formulation is extended to Hilbert-valued feature maps on product matrix spaces, yielding a blockwise Muon probability flow applicable to smooth transformer mixture-of-experts models, thus broadening the optimizer’s theoretical applicability to modern architectures.