Study of 2,497 cardiac patients reveals hidden AI failure when treatment masks outcomes

Clinical ML models usually assume the outcome of interest is cleanly observed for every patient. But in practice, treatment decisions can make the clinically relevant outcome permanently unobservable — a problem the authors call treatment-induced label indeterminacy. Using a cohort of 2,497 post-cardiac-arrest patients, they found 1,429 cases where treatment made neurological outcomes indeterminate. Independent clinical experts provided counterfactual outcome guesses for these uncertain cases, while the remaining 1,068 were certain cases with observed outcomes.

To tackle this, Shen et al. propose a framework that evaluates models separately on certain and uncertain cases, then introduce a simple neural model that trains on both label types with an explicit tradeoff parameter. Results show models with nearly identical certain-case AUROC can differ dramatically in Brier score and in probability estimates for uncertain cases. Improving alignment on uncertain cases generally worsens certain-case accuracy — a tradeoff conventional metrics hide. This matters because prognostication models are most needed for the very patients whose outcomes are obscured by treatment decisions.