arXiv study: Adversarial training's statistical cost fixed via noise-debiased RKHS method

Researchers Yiling Xie and Xiaoming Huo have released a new theoretical study on arXiv (2607.27995) that quantifies the hidden cost of adversarial training in nonparametric settings. Working within the reproducing kernel Hilbert space (RKHS) framework, they derive source-uniform generalization error bounds that explicitly depend on the robustness level, sample size, and kernel spectrum. On a fixed polynomial-spectrum model, they prove a matching lower bound showing that optimally balanced adversarial training converges slower than the minimax prediction benchmark — revealing a fundamental loss of statistical accuracy.

The culprits are not just adversarial perturbations. The paper shows the degradation arises from the interaction between robustness and observation noise: the noise contribution in the mixed robustness term slows approximation, although it paradoxically reduces estimation complexity. To fix this, the authors propose a two-stage noise-debiased procedure that estimates and subtracts the noise contribution from the mixed term. This estimator achieves the minimax polynomial rate up to a logarithmic factor when the robustness level is chosen at the sample-dependent order. Numerical experiments confirm the theoretical findings, offering a principled path to balance robustness and generalization in kernel-based models.