PPAT: New method slashes labeling costs for AI risk estimation

Kianoosh Ashouritaklimi, Valentin Kilian, Daolang Huang, Tom Rainforth, and François Caron from Oxford and Imperial College have proposed Prediction-Powered Active Testing (PPAT), a new label-efficient risk estimation framework. Active testing traditionally selects which test points to label adaptively, but existing estimators don't leverage the informative predictions from powerful black-box models like neural networks. PPAT addresses this by combining the unbiased LURE estimator with a prediction-powered control variate: instead of using predictions as biased pseudo-labels, it uses them to residualize the loss function. This preserves unbiasedness while significantly reducing variance, meaning fewer labels are needed to achieve the same estimation accuracy.

PPAT doesn't just improve the estimator—it also changes which points should be acquired. The authors derive oracle and practical surrogate-based acquisition rules tailored to minimize the variance of their new estimator. They further establish asymptotic normality for PPAT, yielding asymptotically valid confidence intervals and principled uncertainty estimates. Across tabular regression and image classification benchmarks, PPAT outperforms existing active testing methods, with confidence intervals attaining target coverage using substantially fewer labels (often 30-50% less) and smaller interval widths. This work promises to accelerate model evaluation in domains where labeled data is expensive, such as medical imaging, autonomous driving, and scientific discovery.