New 'Range Penalization' Method Boosts Federated Learning Efficiency and Accuracy
A novel regularization technique could save bandwidth and improve model accuracy in federated learning.
This paper introduces range regularization for federated learning with linear systematic components to enhance statistical accuracy and induce cross-client regularity conducive to quantization, coding, and resource efficiency. The method identifies features with shared weights across clients and adaptively clusters the weights of personalized features at extreme values—a process called polar clustering. New proof techniques enable nonasymptotic analysis of statistical accuracy and faithful pattern recovery, and a fast optimization algorithm leveraging varying degrees of local strong convexity reduces iteration complexity. Experiments support the efficacy and efficiency of the proposed approach.
- Introduces 'Range Penalization' (RangeReg) with polar clustering to group personalized weights at extremes
- Provides nonasymptotic theoretical analysis despite regularizer's seminorm nature
- Fast optimization algorithm leveraging local strong convexity reduces iteration complexity
Why It Matters
Makes federated learning more communication-efficient while improving accuracy, enabling better deployment on resource-constrained devices.