Forget Hand-Crafted Sparsity: ZIG-EDA Automates Sparse Black-Box Optimization with Zero-Inflated Gaussians
New EDA jointly optimizes sparsity and active values without hand-crafted operators.
Estimation-of-distribution algorithms (EDAs) are a class of evolutionary methods for black-box optimization that sidestep hand-designed mutation and crossover operators by fitting a probability distribution to the best individuals. However, they have not previously generalized to sparse parameter spaces, where most coefficients of a good solution are exactly zero. Existing sparse black-box optimizers rely on hand-crafted sparsity operators, bi-level schemes, or zeroing thresholds—exactly what EDAs were designed to avoid.
To close this gap, researchers from (affiliation not given) introduce multivariate zero-inflated Gaussian (ZIG) distributions as EDA sampling laws. A latent Gaussian model with separate indicator and value dimensions represents sparsity patterns, correlations among active parameters, and interactions between the two, enabling joint optimization. The latent parameters are identifiable from observed samples, and the authors provide practical amortized inversion-based estimators. On the Lunar Lander benchmark, the resulting ZIG-EDA converged faster and reached higher final returns than a dense Gaussian EDA, a hand-crafted sparse evolutionary algorithm, and an ad-hoc sparse EDA, while finding controllers with only a small fraction of parameters active.
- ZIG-EDA uses multivariate zero-inflated Gaussian distributions to jointly model sparsity patterns and active parameter correlations.
- The latent parameters are identifiable from samples, enabling efficient amortized inversion-based estimators.
- On Lunar Lander, ZIG-EDA outperformed three baselines in convergence speed and final returns, using far fewer active parameters.
Why It Matters
Better sparse optimization reduces computational costs and improves AI training efficiency for real-world black-box problems.