DELTA-TTS turns AR TTS models into 3.3x faster diffusion models
DELTA-TTS converts autoregressive TTS into diffusion models, cutting WER by 1.75% and boosting speed 3.3x
A team from Korean institutions led by Junwon Moon has developed **DELTA-TTS**, a novel framework that converts pretrained autoregressive (AR) text-to-speech (TTS) models into discrete diffusion language models (dLLMs) using lightweight LoRA adaptations.
The core innovation lies in confidence-ordered speech-token decoding, which defers low-confidence tokens to later steps via a 1/t-weighted training objective and time-shifted inference schedule. Adding a convolution module to capture local acoustic context further sharpens text-speech alignment. Trained on just 585 hours of LibriTTS data, DELTA-TTS achieves a 1.75% word error rate (WER) on the Seed-TTS benchmark—outperforming its AR backbone—while generating tokens **3.3x faster**. The diffusion approach also mitigates the error propagation and hallucinations common in traditional AR TTS systems.
- DELTA-TTS (by Junwon Moon et al.) adapts AR TTS models into diffusion models using LoRA, achieving 3.3x faster inference
- Trained on 585 hours of LibriTTS, it reduces WER to 1.75% on Seed-TTS and improves text-speech alignment
- Diffusion-based decoding defers low-confidence tokens, reducing hallucinations vs. AR baselines
Why It Matters
DELTA-TTS unlocks high-quality, faster TTS synthesis while reducing errors—critical for real-time applications like voice assistants and audiobooks.