New arXiv paper reveals optimal subsampling for faster, greener ML training

In a new paper on arXiv, Eyal Vayness and Maxime Sangnier tackle a classic big-data problem: how to train models on a representative subset of data without losing accuracy. Their work focuses on nonparametric supervised learning within reproducing kernel Hilbert spaces (RKHS), a powerful framework used in kernel methods like SVMs and Gaussian processes. The key innovation is a subsampling scheme that selects data points in a principled way—minimizing the trace of the covariance operator—and then adjusts the empirical risk using Horvitz-Thompson weighting. This reweighting corrects for the non-uniform selection probabilities.

The authors prove that their scheme is asymptotically optimal, and they provide a plug-in method that works without prior knowledge of the data distribution. Numerical experiments on both synthetic and real-world datasets confirm that the method reduces computational time and energy consumption without degrading model performance. For practitioners, this means faster training cycles and a smaller carbon footprint—critical as ML models grow ever larger. The paper is especially relevant for teams working with kernel methods at scale, offering a theoretically grounded way to subsample smartly.