CRDA data augmentation cuts MLP regression MSE by 22.9%

Real-world regression often struggles with limited training data, high collection costs, and noisy observations. To address this, a team led by Hossein Mohebbi (Simon Fraser University) introduces Counterfactual Residual Data Augmentation (CRDA), a novel technique specifically designed for tabular regression. The key insight: once a regressor captures the systematic component of the data, the remaining noise forms an invariant residual that stays stable under small perturbations of carefully selected features. CRDA exploits this residual invariance to generate new, realistic training samples without needing additional real data.

The method is model-agnostic, meaning it works with any regressor (e.g., MLP, XGBoost). In experiments across multiple benchmark datasets, CRDA reduced MLP Regressor MSE by an average of 22.9% and XGBoost Regressor MSE by 6.4%. Compared to existing state-of-the-art data generators and augmentation techniques, CRDA consistently achieved the greatest MSE reduction. The approach offers a simple, efficient remedy for noise-prone, small-sample regression settings.

Accepted at the 43rd International Conference on Machine Learning (ICML 2026), CRDA represents a significant step forward for data augmentation in tabular domains, which have lagged behind image and language fields. By adding principled counterfactual variations to training data, it enables more robust model training without requiring expensive or extensive real-world data collection.