FedVPA-GP: New method personalizes LLMs to each user without data sharing
A federated approach that disentangles conflicting preferences like helpfulness vs harmlessness
Deep Dive
Researchers introduce FedVPA-GP, a framework that personalizes large language models in federated settings without centralizing user data. It uses a Federated Mixture Prior and Orthogonal Loss to overcome posterior collapse from local data scarcity. On the HH-RLHF dataset, FedVPA-GP significantly outperforms monolithic baselines, successfully disentangling conflicting intents (e.g., helpfulness vs harmlessness) and enabling dynamic preference switching.
Key Points
- FedVPA-GP solves the 'one-size-fits-all' problem in federated LLM alignment by disentangling conflicting user preferences (e.g., helpfulness vs. harmlessness).
- Introduced a Federated Mixture Prior that stabilizes variational inference when local data is scarce and heterogeneous.
- Orthogonal Loss enforces separation of preference prototypes, enabling dynamic preference switching at inference time on the HH-RLHF dataset.
Why It Matters
Enables truly personalized, privacy-preserving AI assistants that respect diverse and conflicting user values.