New arXiv study reveals when synthetic data helps imbalanced classification

Synthetic data augmentation is widely used to tackle class imbalance in machine learning, but its theoretical underpinnings have been murky. A new paper from Zhengchi Ma, Pengfei Lyu, and Anru R. Zhang, posted on arXiv, provides a rigorous framework to characterize when synthetic minority augmentation actually helps performance. The authors focus on threshold-integrated and threshold-optimized metrics such as AUROC, AUPRC, best-threshold balanced accuracy, and best-threshold F1 score. They decompose augmentation's effect into two components: a change in effective class weighting and a discrepancy between the synthetic and true minority distributions. This decomposition allows them to isolate when augmentation is beneficial versus detrimental.

The paper's key insight is that under well-specified score models, the raw estimator already targets the optimal likelihood-ratio ordering for these metrics. Therefore, augmentation cannot provide a fundamental population-level improvement—any gains come only from finite-sample variance reduction, and it may even introduce bias through synthetic distributional error. The authors back this with minimax lower bounds showing the raw estimator achieves optimal metric-regret rates. However, under model misspecification, augmentation plays a qualitatively different role: by changing the effective class balance, it can alter the restricted-class projection and correct ranking errors induced by the raw imbalanced objective. The paper provides explicit improvement bounds quantifying the roles of approximation error, finite-sample estimation error, and synthetic distributional error. Simulation studies confirm limited gains under well-specification and nontrivial but nonmonotone improvements under misspecification.