Two-Timescale Asymptotic Simulations of Hybrid Inclusions with Applications to Stochastic Hybrid Optimization

A new theoretical paper from researchers Max F. Crisafulli and Andrew R. Teel tackles a core challenge in modern optimization: proving that complex, stochastic training algorithms will actually converge to a solution. The work focuses on 'hybrid inclusions,' a mathematical model that combines continuous dynamics (like gradient flow) with discrete jumps (like parameter updates). This hybrid structure is fundamental to how many AI models are trained today, where learning happens through iterative steps with random variations.

The authors' key contribution is developing 'two-timescale asymptotic simulations' to analyze these systems. They establish sufficient mathematical conditions under which a sequence of algorithm iterations and step sizes will behave predictably in the limit, with its long-term behavior characterized by invariant sets of a simplified 'boundary layer' and 'reduced' system. The paper then applies this abstract theory to a concrete problem: proving that a two-timescale stochastic approximation of a hybrid optimization algorithm will asymptotically recover the behavior of its deterministic version. This provides a rigorous foundation for trusting that noisy, real-world training runs will converge to the same solution as idealized theoretical models.