Adaptive Bayesian Expert Aggregation Improves Online Learning
New framework treats Bayesian update rules as experts for adaptive predictions.
Jun and Ohn address a key limitation of Bayesian online learning: performance depends on fixed inferential choices like learning rates and priors. Their framework treats each Bayesian update rule as an expert and aggregates them based on per-round predictive losses. They prove the aggregate competes with the best expert in hindsight, with aggregation cost determined by how each expert's performance is evaluated.
Instantiated in online conformal inference, the method yields a smoothed Bayesian counterpart of adaptive conformal inference with long-run randomized coverage. For Gaussian process regression, it achieves an oracle inequality in cumulative predictive Kullback-Leibler risk and adapts to unknown Hölder smoothness up to logarithmic factors. Experiments confirm the aggregate tracks strong experts without manual selection, offering a robust, adaptive approach to online Bayesian learning.
- Treats Bayesian update rules as experts and aggregates them via sequential predictive losses.
- Proves the aggregate competes with the best expert in hindsight with bounded aggregation cost.
- Achieves oracle inequality in cumulative KL risk for Gaussian process regression, adapting to unknown smoothness.
Why It Matters
Enables adaptive, uncertainty-aware online predictions without manual tuning of learning rates or priors.