chi_sao: GPU-native optimizer achieves 100% mode recovery, up to 39x speedup

Ira Wolfson's chi_sao (Convergence-Halt-Invert-Stick-And-Oscillate) is a GPU-native population optimizer designed for the notoriously hard problem of finding all modes in multimodal black-box functions. Existing approaches like basin-hopping, CMA-ES, and multistart gradient descent operate sequentially and cannot exploit modern GPU parallelism. chi_sao runs an entire sample batch simultaneously on the GPU, achieving a striking 100% mode recovery on all 42 functions of the Simon Fraser University optimization benchmark suite across dimensions d=2 to d=64. On the hardest multimodal functions, CPU baselines collapse at d≥8; chi_sao continues to recover all modes. Speedups reach 34x over basin-hopping on Michalewicz d=64 and 39x on unimodal Rotated Hyper-Ellipsoid d=64. The algorithm is robust to substantial likelihood noise (σ_noise up to 1.0) and is available as an open-source Python package on PyPI.

The core innovation is an asymmetric convergence-anticonvergence oscillation cycle: samples that reach true peaks are frozen (“stuck”) and preserved, while the rest keep exploring via momentum-based anti-convergence and stochastically smoothed gradients. Adaptive reseeding strategies, Repulse Monkey and Golden Rooster, maintain population diversity. All benchmarks evaluate the objective by value alone using finite differences, so the reported speedups represent a derivative-free worst case. This work offers a practical tool for Bayesian inference, hyperparameter tuning, and scientific computing where finding multiple optima in high-dimensional black-box functions is critical—now dramatically faster on GPU hardware.