Joint ARD method prunes features and outliers in one Bayesian step
ICML 2026 paper from Timans et al. makes sparse recovery robust to data contamination.
Sparse Bayesian Learning (ARD) has long been a staple for feature selection in linear systems, but its assumption of homoscedastic noise makes it brittle when data contains outliers or misspecifications. In a paper published at ICML 2026, Alexander Timans and co-authors introduce a symmetric pruning framework that jointly optimizes relevance weights for both features and individual data points. By extending the marginal likelihood objective, the method learns which samples to discard alongside which predictors to keep, effectively performing robust regression and model sparsification in a single pass.
The key technical insight is that the new formulation preserves conjugacy and admits closed-form update rules, making it computationally attractive for standard optimization pipelines. Empirical results across eight regression datasets show that joint ARD consistently outperforms standard Sparse Bayesian Learning and popular robust baselines, achieving sparser models with lower prediction error. The approach also aligns with influence function theory, providing a principled Bayesian justification for outlier removal. This work opens the door to more resilient sparse recovery in high-dimensional settings where both feature selection and data quality are critical.
- Jointly prunes irrelevant features and outliers using a single Bayesian marginal likelihood objective
- Preserves conjugacy and enables closed-form updates, unlike many robust regression extensions
- Demonstrated across eight regression tasks, achieving sparser and more robust predictions than standard Sparse Bayesian Learning
Why It Matters
A practical extension to ARD that automatically cleans data while selecting features, improving reliability in noisy real-world applications.