Beyond Expected Information Gain: Stable Bayesian Optimal Experimental Design with Integral Probability Metrics and Plug-and-Play Extensions
Replacing KL divergence with Wasserstein distance yields geometry-aware stability in Bayesian optimal experimental design.
In a new paper on arXiv, researchers Di Wu, Ling Liang, and Haizhao Yang address fundamental bottlenecks in Bayesian Optimal Experimental Design (BOED), a framework for decision-making in resource-constrained data acquisition. Traditional BOED selects designs by maximizing expected information gain (EIG) using Kullback-Leibler (KL) divergence, but this approach suffers from support mismatch, tail underestimation, and rare-event sensitivity. The nested expectations required for EIG evaluation are computationally challenging, and even advanced variational methods leave the underlying log-density-ratio objective unchanged.
The authors propose replacing density-based divergences with integral probability metrics (IPMs), including the Wasserstein distance, Maximum Mean Discrepancy (MMD), and Energy Distance, creating a flexible plug-and-play BOED framework. They provide theoretical guarantees showing IPM-based utilities offer stronger geometry-aware stability under surrogate-model error and prior misspecification compared to classical EIG. Empirically, IPM-based designs yield highly concentrated credible sets. The framework also extends to geometry-aware discrepancies beyond IPMs, demonstrated with a neural optimal transport estimator, achieving accurate optimal designs in high-dimensional settings where conventional nested Monte Carlo and variational methods fail.
- Replaces KL divergence with IPMs (Wasserstein distance, MMD, Energy Distance) for stable BOED.
- Provides theoretical guarantees of geometry-aware stability under model errors and prior misspecification.
- Extends to neural optimal transport for accurate designs in high-dimensional settings.
Why It Matters
This framework makes experimental design more robust and scalable, critical for resource-constrained data acquisition in science and engineering.