New paper shows how to aggregate calibrated forecasts better

Decision-makers often rely on multiple probabilistic forecasts that are individually calibrated but not fully informative. A new paper by Xinxiang Guo, Yingkai Li, and Yifen Mu develops a framework for aggregating such forecasts when only calibration is known. They prove that the joint distribution of calibrated forecasts contains decision-relevant information that no single expert provides, meaning the standard optimal-in-hindsight (OIH) benchmark can substantially understate attainable performance. To address this, they introduce a robust max-min benchmark: the best payoff a decision-maker can guarantee against all profile-wise conditional-mean mappings compatible with calibration. This benchmark is tractable via a linear-programming formulation and dominates OIH up to calibration error.

Importantly, the robust benchmark remains below the full Bayesian benchmark, clarifying the value of knowing experts' information structures. The authors also provide online algorithms that achieve the robust benchmark under only forecast feedback, and stronger contextual benchmarks when state feedback is available. This work bridges theoretical economics and game theory, offering practical tools for aggregating forecasts in settings like climate modeling, financial risk, or AI safety where multiple calibrated models exist but their internal knowledge is unknown.