Spectral-Transport Stability and Benign Overfitting in Interpolating Learning

A team of researchers has published a significant theoretical paper titled 'Spectral-Transport Stability and Benign Overfitting in Interpolating Learning' on arXiv. The 50-page work tackles a central puzzle in modern machine learning: why highly overparameterized models—those with far more parameters than training examples—can achieve zero training error (overfit) yet still perform well on new data, a phenomenon known as 'benign overfitting.' The authors develop a novel 'spectral-transport stability' framework that explains this by showing excess risk is controlled by three factors: the spectral geometry of the data, the sensitivity of the learning rule to single data point changes, and the alignment structure of label noise.

This framework leads to the creation of a 'scale-dependent Fredriksson index,' a single complexity parameter that combines effective dimension, transport stability, and noise alignment. The researchers prove finite-sample risk bounds and establish a sharp criterion for when overfitting becomes benign versus destructive. They derive explicit phase-transition rates under polynomial spectral decay and provide a model-specific theorem for polynomial-spectrum linear interpolation. Crucially, the work connects several key concepts—algorithmic stability, the double descent curve, benign overfitting, and the implicit bias of optimization algorithms—into a unified theoretical account. This provides a structural explanation for the empirical success of modern interpolating estimators, like large neural networks, that defy classical statistical wisdom.