New Confidence-Ranking Method Cuts Evaluations in Noisy Black-Box Optimization
Researchers slash computational cost with adaptive sampling that outperforms state-of-the-art CMA-ES.
Real-world optimization problems often involve black-box functions with noisy evaluations, a challenge known as noisy optimization problems (NOPs). Traditional evolutionary algorithms like CMA-ES and genetic algorithms (GA) rely on implicit averaging (large populations) or explicit averaging (resampling). Halim, Singh, and Ray show that implicit averaging fails under heteroscedastic noise—where variance changes across the search space. Their new method uses computationally efficient explicit averaging with adaptive sampling, dynamically allocating samples per solution based on confidence thresholds to reduce total evaluations without sacrificing ranking accuracy.
The method is demonstrated within both CMA-ES and GA frameworks, proving versatility. Extensive testing on newly introduced heteroscedastic test problems shows it significantly outperforms state-of-the-art NOP solvers. For practitioners, this means faster, more reliable optimization in engineering, finance, and science where evaluations are expensive and noise unpredictable. The paper provides code and data, offering a practical path for real-world black-box optimization under realistic noise conditions.
- Introduces heteroscedastic test problems that better reflect real-world noise than homoscedastic benchmarks.
- CRAS adapts sampling budget per solution using confidence thresholds, reducing total evaluations significantly.
- Works with both CMA-ES and GA frameworks, demonstrating broad applicability across evolutionary algorithms.
Why It Matters
Faster, cheaper optimization for expensive real-world problems with unpredictable noise—a game changer for engineering and ML.