ALAS: New Kernel Family Lets Bayesian Optimization Automatically Adapt to Data Smoothness

Bayesian Optimization (BO) is essential for optimizing expensive black-box functions, but its performance heavily depends on choosing a kernel that matches the objective's unknown structure. Researchers Weibo Huang and Cheng Hua introduce ALAS (Additive Learnable Alpha-Stable Kernels), a novel Gaussian Process kernel family that sidesteps this issue by learning the kernel's smoothness directly from data. ALAS is built from symmetric α-stable spectral components, where the stability parameter α governs tail heaviness and smoothness. By learning α during optimization, ALAS can capture both smooth trends and sharp irregularities, adapting to the objective's true nature without manual tuning. This is a significant step forward from traditional kernels like RBF or Matérn, which have fixed smoothness assumptions.

The authors present two parameterizations: ALAS, a single stationary component with joint spectral modulation, and ALAS-Sep, a separable variant that learns dimension-wise tail behavior, improving robustness on approximately decomposable objectives. ALAS-Sep is particularly useful when different input dimensions have varying levels of smoothness. Experiments on standard benchmarks (e.g., synthetic functions) and real-world surrogate problems (e.g., hyperparameter tuning) demonstrate that ALAS consistently achieves strong and robust performance across diverse settings, often outperforming fixed kernels. The work is available on arXiv (2607.18282). For practitioners, ALAS reduces the need for kernel selection expertise, making BO more accessible and reliable. Future work may extend to high-dimensional or dynamic settings.