Decision-calibrated conformal method slashes ad pacing uncertainty by 99%

Pacing decisions in streaming advertising rely on uncertain forecasts of future inventory, demand pressure, incremental response, and member-experience load. Traditional conformal calibration methods measure forecast error generically, often leading to overly conservative uncertainty estimates that waste advertising budget or miss delivery goals. This new paper from Shekhar and Howard introduces a decision-calibrated conformal framework that directly measures forecast error by its largest impact on the pacing policies that could actually be deployed. The key theorem proves that the proposed score is the smallest valid uncertainty measure uniformly protecting all deployable pacing policies, geometrically represented as the support function of the signed policy sensitivity set.

Applying split conformal calibration, the method achieves finite-sample coverage. On two public datasets (Criteo Uplift and KuaiRand), results are dramatic: traditional conformal pacing gave high residual radii of 7,236.7 (Criteo) and 4,629.4 (KuaiRand). The proposed decision-calibrated approach reduced these to 18.4 and 278.6 respectively, with separate margins for value, delivery, budget, and member load. On Criteo, the method certified a less aggressive pacing policy than the point-forecast baseline, cutting held-out any-violation rate from 16.7% to 3.3%—with zero budget and member-load violations. The paper establishes that forecasts and response estimates should be judged by whether they shrink the uncertainty that the pacing decision uses, leading to confident, practical decisions.