New proof shows non-affine aggregation breaks convex learning monotonicity

A new theoretical result from Boudou, Le Bars, Gupta, and Bellet (arXiv:2606.28123) tackles a fundamental question in convex optimization: can non-affine gradient aggregation maintain the monotonicity needed for convergence? The answer is a definitive no. The authors prove that monotonicity of the update operator holds **if and only if** the aggregation rule is positively affine (i.e., a weighted average plus a non-negative scaling factor). This means that common non-affine techniques—such as gradient clipping for robustness, differentially private mechanisms, adaptive learning rates, or fairness constraints—inherently break the monotonicity guarantee, leading to erratic convergence or even divergence. The proof is rigorous and extends to any first-order convex learning algorithm that uses aggregated gradient updates.

Beyond the negative result, the paper quantifies the stability degradation: non-affine aggregation can cause the iterates to oscillate or fail to converge to the optimum, even in convex settings where standard linear averaging would guarantee convergence. The authors also propose a path forward by identifying sufficient conditions—such as contractive operators or specific regularizers—that can restore monotonicity. This work provides a unifying theoretical framework that explains disparate failure modes observed in modern systems that enforce constraints like privacy, fairness, or robustness. For practitioners, it suggests that adding such constraints requires careful redesign of the aggregation step or relaxing convergence guarantees.