This AI Optimization Technique Isn't Just Faster — It Solves a Longstanding Parallel Performance Problem

Probabilistic model-based evolutionary algorithms are powerful for black-box optimization, but their efficiency hinges on two hyperparameters: learning rate and weight parameters. While ASNG (Adaptive Stochastic Natural Gradient) already adapts the learning rate, weight parameters remain static, causing suboptimal performance as population sizes grow—a critical issue for parallel evaluation of time-consuming tasks. To address this, Yutaro Yamada, Kento Uchida, and Shinichi Shirakawa introduce WA-ASNG (Weight Adaptation ASNG). The algorithm computes an estimated signal of the update direction from accumulated natural gradients, then uses gradient ascent to continuously adjust weight parameters, thereby maximizing the expected improvement in the objective. This complements the existing learning-rate adaptation, which ensures monotonic improvement, by further accelerating convergence.

Experimental results on binary optimization problems demonstrate WA-ASNG's superiority across a wide range of population sizes (25–100). It consistently outperforms both the classic PBIL and the original ASNG, achieving faster convergence and higher final solution quality. Notably, WA-ASNG maintains robust performance even under strong noise, a practical advantage for real-world black-box optimization. The paper, accepted at EvoCOP 2026, includes open-source code on GitHub. This work is particularly relevant for engineers running large-scale parallel evaluations, as it squeezes more performance from distributed computing without manual hyperparameter tuning.