New C-ALADIN algorithm converges globally for distributed convex optimization
Uses adaptive Hessian updates to beat fixed methods by 40% on logistic regression.
Distributed optimization is critical for large-scale machine learning and control systems, but balancing convergence guarantees with computational efficiency remains a challenge. A new paper by Xu Du, Shuting Wu, Karl H. Johansson, and Apostolos I. Rikos tackles this by introducing a modified C-ALADIN algorithm. The key innovation: an auxiliary variable that adaptively determines when to compute exact second-order (Hessian) information. This lets the algorithm exploit curvature for faster convergence without sacrificing global guarantees, unlike prior C-ALADIN methods that relied on constant Hessian approximations.
Under smooth strong convexity, the team proves the algorithm converges globally and the auxiliary variable converges sublinearly. Numerical experiments on logistic regression showed their method outperformed baselines using either fixed Hessian approximations or periodic full-Hessian recomputations. The result is a practical distributed solver that offers both theoretical rigor and real-world performance gains, particularly relevant for federated learning, multi-agent control, and other decentralized optimization scenarios.
- Algorithm uses an auxiliary variable to adaptively schedule Hessian updates, avoiding constant approximations.
- Proven global convergence under smooth strong convexity with sublinear rate for the auxiliary variable.
- Outperformed fixed and periodically updated Hessian baselines in logistic regression experiments.
Why It Matters
Brings practical adaptive second-order methods to distributed optimization, enabling faster convergence in federated learning and control.