FedSPC boosts personalized federated learning with shared parameter correction

Personalized federated learning (PFL) splits model parameters into shared and personalized parts to handle statistical heterogeneity across clients. However, shared parameters are updated by clients optimizing different local objectives, causing inconsistent updates that weaken the shared representation. FedSPC (Federated Shared Parameter Correction) addresses this by applying a control-variate correction exclusively to shared parameters, leaving personalized ones untouched. This modular approach integrates seamlessly into three common PFL settings: shared feature extractors, shared classifiers, and fully shared models with local regularization. The correction stabilizes training without requiring changes to the underlying PFL algorithm.

Experiments on CIFAR-100 and Tiny-ImageNet using ViT, ResNet-34, and VGG-11 show consistent accuracy improvements across five representative PFL methods: FedPer, FedRep, FedBABU, LG-FedAvg, and Ditto. The method requires no additional communication rounds and adds minimal computational overhead. FedSPC's plug-and-play nature makes it a practical upgrade for any PFL pipeline, directly tackling the shared parameter drift problem. Accepted at FL@FM-IJCAI'26, this work provides a simple yet effective fix for a core optimization challenge in federated learning.