Freiesleben et al.: Explainable AI fails to let users challenge wrong decisions

A new paper from Timo Freiesleben, Kristof Meding, and Gunnar König challenges the prevailing focus of explainable AI (XAI) on algorithmic recourse—the practice of showing users how to change their features to obtain a favorable decision. The authors argue that this approach assumes the decision is valid, whereas a more pressing ethical and legal problem is algorithmic contestability: enabling individuals to review and overturn incorrect decisions made by opaque machine learning systems.

To operationalize contestability, the paper distinguishes it clearly from recourse. Recourse presumes the decision is correct and offers pathways to change it. Contestability starts from the presumption that the decision may be wrong and seeks evidence to challenge it. The authors demonstrate that standard XAI methods like counterfactuals, LIME, or Anchors only reveal errors near the individual's input, failing to provide grounds to overturn the decision. They instead identify three types of evidence that can render a decision normatively indefensible: predictive multiplicity (different models giving different results), incorrect feature values (data entry errors), and neglected overruling evidence (existing information that undermines the decision). The paper concludes by analyzing how current EU legislation already grants individuals some legal rights to these forms of evidence, suggesting a path toward more contestable AI systems.