Meta-LoRA personalizes LLMs across domains with 47.9% less degradation
New Meta-LoRA method cuts cross-domain LLM personalization errors by 47.9%...
A new arXiv paper proposes PAC-Bayes-regularized Meta-LoRA for cross-domain LLM personalization. It uses a meta-learned LoRA initialization and adjusts update strength based on evidence quality, helping avoid overfitting when user data is sparse. The method also separates preferences into user and domain components. On the HiCUPID benchmark, it cuts cross-domain win-rate degradation by 47.9% relative to the best competing baseline and boosts win rate by 110.2% for unseen-user cold starts.
- Meta-LoRA reduces cross-domain LLM personalization degradation by 47.9% on HiCUPID benchmarks
- Improves unseen-user cold start performance by 110.2% through adaptive evidence calibration
- Combines PAC-Bayes regularization with meta-learning to prevent overfitting while enabling stronger personalization as evidence grows
Why It Matters
Enables LLMs to maintain consistent personality across domains with minimal fine-tuning, crucial for enterprise and consumer applications requiring coherent multi-domain interactions.