GLOBE: New coreset method slashes training data and boosts accuracy

On-device training of deep neural networks faces strict computational and memory limits, so coreset selection—retaining only a compact subset of real training samples—is a practical solution. But existing gradient-based methods often rely on gradients from a single model snapshot and use greedy or pursuit-based selection, which fail to capture how optimization dynamics evolve and struggle with strongly correlated samples. A new paper from Hetian Liu and colleagues at multiple institutions introduces GLOBE (Gradient Local-Balanced Extraction), a trajectory-aligned coreset selection framework that reframes sample selection as a globally optimized sparse weighting problem.

GLOBE represents each sample by a gradient trajectory built from multiple training checkpoints, capturing influence across different optimization stages. A multi-order matching objective aligns the first-order mean and projected uncentered second-order moments of gradient trajectories, preserving the full dataset's training behavior. The framework combines Group LASSO, Elastic Net regularization, and nonnegative budget constraints to induce group- and sample-level sparsity while stabilizing correlated trajectory weights, followed by class-balanced Top-K selection. In experiments across six benchmarks and five evaluation architectures, GLOBE consistently outperforms existing coreset methods in downstream test accuracy, particularly at low retention ratios—demonstrating that combining dynamic gradient information, multi-order distribution matching, and structured sparsity enables data-efficient learning that is practical for on-device AI.