Merging Methods for Multilingual LLM Editing: Study Reveals Best Strategy

A new empirical study from researchers at POSTECH investigates merging methods for multilingual knowledge editing (MKE) in large language models. The work, submitted to arXiv, tests six vector merging variants across two popular backbone LLMs and 12 languages on the MzsRE benchmark under a large-scale batch-editing setting. The core challenge in MKE is that language-specific edits interfere with each other, even though locate-then-edit methods work well in monolingual settings. The study focuses on the effectiveness of vector merging, the ability of Task Singular Vectors for Merging (TSVM) to reduce multilingual interference, and the impact of weight scaling factor and rank compression ratio on performance.

The results show that vector summation with shared covariance is the most reliable overall strategy, while simple summation without shared covariance performs poorly. TSVM improves performance in some settings but its ability to mitigate multilingual interference is limited. The researchers also found that performance is sensitive to both weight scale and rank ratio—larger-than-default scaling and relatively low rank often yield better results. These findings clarify the practical strengths and limits of current vector merging methods for MKE, providing guidance for future research in multilingual knowledge editing.