MinShap: A Modified Shapley Value Approach for Feature Selection
New method replaces averages with minimum contributions to isolate direct feature effects, outperforming LOCO and Lasso.
Researchers Chenghui Zheng and Garvesh Raskutti have introduced MinShap, a novel algorithm that adapts the cooperative game theory concept of Shapley values to the persistent challenge of feature selection in machine learning. Traditional Shapley values, while excellent for feature attribution in complex, non-linear models with dependent features, conflate a feature's direct effect on a model's output with its indirect effects mediated through other features. This makes them suboptimal for selecting which features are truly essential. MinShap addresses this by fundamentally altering the aggregation rule: instead of calculating the average marginal contribution of a feature across all possible orderings (permutations) of the feature set, it considers the *minimum* marginal contribution. This shift is theoretically motivated by assumptions in directed acyclic graphical (DAG) models and is designed to be more faithful in identifying only features with a direct causal link to the response.
The paper establishes a strong theoretical foundation for MinShap, including a guarantee for controlling Type I error (the chance of incorrectly selecting an irrelevant feature). In comprehensive testing, the authors demonstrate that MinShap outperforms contemporary feature selection methods like LOCO (Leave-One-Covariate-Out), the Generalized Covariance Measure (GCM), and even the classic Lasso regression in terms of both accuracy in selecting the correct features and the stability of those selections across different data samples. Furthermore, the researchers introduce a related suite of algorithms that leverage a multiple testing or p-value perspective, which are shown to enhance performance in settings with lower sample sizes, accompanied by their own supporting theoretical guarantees. This work provides a powerful, principled bridge between explainable AI attribution techniques and robust feature selection pipelines.
- MinShap modifies Shapley values by using the minimum marginal contribution across permutations, not the average, to isolate direct feature effects.
- The method provides theoretical Type I error control guarantees and outperforms LOCO, GCM, and Lasso in accuracy and stability in simulations.
- A related suite of p-value-based algorithms is introduced for improved performance in lower-sample-size scenarios.
Why It Matters
Provides data scientists with a more robust, explainable method for selecting the most causally relevant features in complex models, improving model interpretability and performance.