Subspace Acquisition Functions Enable Scalable Batch Bayesian Optimization
New method draws axis-aligned subspaces to pick points, dramatically speeding convergence at large batch sizes.
Bayesian optimization (BO) is widely used for hyperparameter tuning and experimental design, but extending it to batch evaluation (running multiple points in parallel) has been hampered by poor scalability: as batch size grows, optimization efficiency often plummets. To address this, Dawei Zhan, Zhaoxi Zeng, Shuoxiao Wei, and Ping Wu (published in ACM Transactions on Evolutionary Learning and Optimization, 2026) introduce a simple yet effective approach called subspace acquisition functions.
The idea is elegantly straightforward: instead of trying to select a batch of points from the full high-dimensional space using a single acquisition function (which becomes increasingly complex and inefficient), the algorithm draws several axis-aligned subspaces (each retaining only a subset of dimensions). It then applies any standard acquisition function (e.g., expected improvement) independently on each subspace to pick one point. This reduces the combinatorial complexity and naturally scales to large batch sizes. Numerical evaluations demonstrate that the method converges faster than sequential BO and matches or exceeds ten state-of-the-art batch BO algorithms, all while requiring no additional hyperparameter tuning. The implementation is available on GitHub.
- Proposes subspace acquisition functions where each batch point is selected from a different axis-aligned subspace of the original problem.
- Achieves significant convergence speedups over sequential Bayesian optimization, especially at large batch sizes.
- Competes with or outperforms ten existing batch BO algorithms across multiple benchmarks, with open-source code released.
Why It Matters
Enables efficient parallel hyperparameter tuning for large-scale ML, reducing wall-clock time for model development.