New quantile mapping algorithm makes reinforcement learning counterfactually fair in healthcare

A team of researchers (Jianhan Zhang, Jitao Wang, John D. Piette, Donglin Zeng, Chengchun Shi, and Zhenke Wu) has published a new approach to making reinforcement learning (RL) fair in high-stakes domains like healthcare. Their paper, arXiv:2608.08743, introduces a data preprocessing algorithm that works alongside policy learning to enforce counterfactual fairness (CF)—a causal framework that ensures outcomes are fair across subpopulations. The algorithm uses a quantile distribution mapping method to recursively estimate counterfactual states and rewards, effectively generalizing common additivity assumptions used in prior counterfactual prediction. This allows the preprocessing step to handle more complex, nonlinear relationships in sequential decision-making.

The authors provide theoretical guarantees: under mild regularity conditions, the per-step counterfactual unfairness and the infinite-horizon suboptimality gap are both bounded. This means the approach can produce RL policies that are provably fair and near-optimal, even over long time horizons. Empirically, they tested the algorithm in numerical simulations and applied it to a real-world interventional digital health dataset, demonstrating its practical viability. The work is particularly relevant for AI-driven healthcare interventions, where RL models might otherwise restrict access to valuable services for certain groups. By making the data itself fair, the method offers a modular, model-agnostic solution that can be integrated into existing RL pipelines.