MeanVC 2 slashes voice conversion latency to 110ms with new chunking technique
Half the latency, better sound quality, and accepts low-quality reference audio
MeanVC 2, presented by Guobin Ma and eight co-authors and accepted at Interspeech 2026, tackles the biggest hurdles in streaming voice conversion: latency and audio quality robustness. The original MeanVC suffered from chunk-wise autoregressive denoising that doubled training sequence length, degraded quality with small chunk sizes, and required clean reference audio. MeanVC 2 introduces future-receptive chunking (FRC), which schedules past and future receptive fields across diffusion transformer decoder layers to eliminate the need for clean-chunk teacher forcing. This enables stable conversion with a 40ms chunk size, slashing end-to-end latency from 211ms to just 110ms.
A second innovation is the universal timbre token encoder. Instead of relying directly on reference mel-spectrograms (which fail on noisy samples), the encoder builds a global speaker embedding and retrieves fine-grained timbre cues via cross-attention. This makes MeanVC 2 robust to low-quality references and improves zero-shot speaker similarity. Audio samples are public, and source code will be released. The combination of FRC and the new encoder means MeanVC 2 is the first model to offer sub‑200ms latency without sacrificing sound quality, making real-time voice conversion in live streaming and telepresence feasible for the first time.
- Latency reduced from 211ms to 110ms – nearly 50% faster – using future-receptive chunking with 40ms chunk size
- New universal timbre token encoder uses global speaker embeddings and cross-attention to handle noisy or low-quality reference audio
- Outperforms MeanVC significantly; accepted at Interspeech 2026 with public audio samples and upcoming open-source code
Why It Matters
Real-time voice conversion with 110ms latency opens doors for live streaming, telepresence, and accessibility tools.