Calibeating Prediction-Powered Inference

Lars van der Laan and Mark van der Laan have introduced a new method called Calibrated Prediction-Powered Inference (Calibeating) for semisupervised mean estimation. This approach addresses a common challenge: when you have a small labeled dataset and a large unlabeled one, and a black-box prediction model that might be miscalibrated. Traditional methods like augmented inverse-probability weighting (AIPW) can be inefficient if the prediction scores don't align well with the actual outcomes. Calibeating post-hoc calibrates these predictions on the labeled sample before using them for estimation, requiring no retraining of the model. The researchers explore both linear and isotonic calibration, with isotonic calibration showing first-order optimality guarantees—it can improve predictive accuracy and estimator efficiency beyond the original score and simpler post-processing methods, and no further processing of the isotonic score yields additional gains.

In their analysis, the authors clarify relationships among existing estimators: the original PPI estimator is a special case of AIPW and can be inefficient when the prediction model is accurate, while PPI++ equates to AIPW with empirical efficiency maximization. Through simulations and real-data experiments, Calibeating often outperforms standard PPI and is competitive with or surpasses AIPW and PPI++. The team has released an accompanying Python package, ppi_aipw, to facilitate adoption. This work, submitted to arXiv on April 23, 2026, spans multiple fields including machine learning, artificial intelligence, econometrics, and quantitative methods, making it broadly relevant for practitioners dealing with limited labeled data.