AI's "Important" Reasons Aren't All the Same, Researchers Warn
The words "this mattered most" can mean five different things — and change the answer.
Researchers Garvesh Raskutti, Kris Sankaran and Jiaxin Ye published a paper developing a unified perspective based on "null importance" — a population-level characterization of when a feature is irrelevant under a specified notion of relevance. The paper argues that the term "importance" encompasses several fundamentally different notions of relevance — arising from marginal and conditional statistical relevance, predictive risk, functional invariance, and causal effects — and that these notions answer different scientific questions. According to the article, the distinction is particularly consequential in two applications: algorithmic fairness, where common fairness criteria correspond to different notions of null importance, and genomic perturbation modeling, where different notions of relevance lead to different conclusions about what a prediction model has learned. The paper also establishes sufficient conditions under which null notions coincide, gives counterexamples showing how they diverge when those conditions fail, and provides simulations and case studies on image and multiomics data. It was submitted to Statistical Science.
- "Feature importance" sounds like one idea but is really at least five — marginal relevance, conditional relevance, prediction risk, functional invariance and causal effect.
- Two reasonable definitions can give opposite answers, especially when factors overlap (like income and postcode) or when key factors aren't measured at all.
- In hiring, lending and gene research, confusing these meanings can make a model look fair — or unfair — depending on which definition was used.
Why It Matters
If you're denied a loan or a job by an AI, the stated "reason" may depend on which definition was used.