New paper finds unavoidable fairness trade-off in conformal prediction

Conformal prediction, a popular framework for producing prediction sets with guaranteed coverage, often calibrates a single threshold across all groups. A new paper by Gao et al. (arXiv:2605.14260) shows that this pooled approach unavoidably hides cross-group differences in score distributions, leading to group-wise coverage distortion. The authors prove a conservation law: the distortion in coverage across groups is bounded below by the cross-group quantile heterogeneity, meaning that some groups will systematically have worse coverage unless the calibration accounts for group membership.

The paper further demonstrates that the two leading fairness definitions for conformal prediction—Equalized Coverage (same coverage rate across groups) and Equalized Set Size (same average set size)—are fundamentally in tension. Any algorithm that tries to enforce one fairness criterion will necessarily violate the other, as the distortion shifts from the coverage dimension to the size dimension. The authors quantify the cost of moving between policies that treat groups separately versus pooling them, and confirm this bidirectional trade-off through experiments on synthetic and real data. These results provide a principled lens for practitioners: calibration choice cannot eliminate cross-group heterogeneity; it only decides where the distortion appears.