Areces, Duchi, Sommers achieve subdifferential stationarity for stochastic convex problems

A new paper by Felipe Areces, John Duchi, and Malo Sommers tackles a long-standing challenge in stochastic convex optimization: guaranteeing that the subdifferential of the objective truly contains a small element at a stationary point. Previous work often settled for weaker surrogates like proximity-to-stationarity or small gradients of the Moreau envelope. The authors argue these are insufficient because subdifferentials of convex functions don't converge uniformly, even near the optimum. Their solution leverages dimension theory to decompose the graph of the subdifferential, showing how stochastic sampling can preserve essential “pieces” of these graphs. This decomposition allows them to apply proximal-point-like methods effectively, achieving the stronger stationarity criterion.

The impact of this work is primarily theoretical, but it has practical implications for the foundations of optimization in machine learning. By proving convergence to a point where the subdifferential contains a small element, the paper offers a more principled guarantee for algorithms used in training models under stochastic noise. The methods introduced—dimension-theoretic decomposition and stochastic graph preservation—may inspire new algorithm designs that achieve this stronger form of stationarity in practice. While the paper is currently a preprint on arXiv (stat.ML, cs.LG, math.OC), it represents a significant step forward in understanding the limits of stochastic convex optimization.