Characterizing Online and Private Learnability under Distributional Constraints via Generalized Smoothness

A team of researchers from MIT and Stanford has published a theoretical breakthrough paper titled 'Characterizing Online and Private Learnability under Distributional Constraints via Generalized Smoothness' on arXiv. The work addresses a fundamental question in learning theory: what minimal assumptions enable learning when data isn't purely independent or adversarial, but comes from adaptively chosen distributions within a fixed family. Building on recent work bridging statistical learning theory extremes, the researchers study sequential decision making under distributional adversaries and provide a near-complete characterization of learnable distribution families.

The paper introduces 'generalized smoothness' as the key condition determining learnability, proving that a distribution family admits VC-dimension-dependent regret bounds for every finite-VC hypothesis class if and only if it is generalized smooth. The researchers provide universal algorithms that achieve low regret under any generalized smooth adversary without explicit knowledge of the distribution family. When the family is known, they offer refined bounds using a combinatorial parameter called the fragmentation number. Crucially, the work extends to differential privacy, showing generalized smoothness also characterizes private learnability under distributional constraints, revealing surprising connections between online learning and privacy-preserving algorithms.