RL method cuts code-switched ASR training data by 90%

A team of researchers from an undisclosed institution (authors Ziwei Ye and Peter Vickers) has introduced a reinforcement learning recipe for adapting audio-language models to code-switched automatic speech recognition (ASR) that dramatically reduces data requirements. Their method, called RLVR (reinforcement learning with verifiable rewards), uses group relative policy optimization (GRPO) combined with two specialized rewards: an error rate reward that penalizes transcription mistakes and a script fidelity reward that penalizes incorrect writing systems (e.g., mixing Latin with Devanagari). They also employ a two-pass draft-and-refinement procedure to improve decoding at language boundaries.

Using Qwen2-Audio as a reproducible testbed across 10 language pairs, the team trained exclusively on TTS-generated code-switched speech. Remarkably, RLVR with only 10% of the training data matched the performance of LoRA-based supervised fine-tuning (SFT) trained on the full dataset. The largest gains were observed on typologically distant language pairs, and the improvements transferred zero-shot to a human-recorded code-switching corpus. The paper has been accepted at Interspeech 2026 and is available on arXiv.