LS-MOPD: New distillation method beats RL teachers in multilingual ASR

Multilingual LLM-based automatic speech recognition (ASR) often struggles because joint modeling of languages with different acoustic and phonological traits creates optimization conflicts. Researchers from a collaborative academic team propose LS-MOPD (Language-Specialized Multi-Teacher On-Policy Distillation) to solve this. The approach decouples language-specific learning from multilingual integration: first, independent language-specialized teachers are optimized using reinforcement learning (RL); then, their expertise is distilled into a single generalist student using language routing and token-level multi-teacher distillation. This prevents direct cross-lingual interference during training.

The paper also investigates static and dynamic acoustic-prefix configurations to study how teacher-student prefix consistency impacts on-policy distillation. Experiments across Mandarin, Mandarin subdialects, Cantonese, and English show that LS-MOPD consistently outperforms RL baselines and even exceeds the empirical performance envelope defined by the best-performing RL teachers. This suggests the student can generalize beyond any single teacher, combining strengths across languages. The work offers a practical path to building better multilingual voice assistants, transcription services, and speech interfaces where one model handles diverse languages with less performance degradation.