DLLM-TTS: Block discrete diffusion model hits 0.15 RTF for fast, accurate speech

Current text-to-speech systems force a painful trade-off: autoregressive codec language models produce highly intelligible speech but demand massive models and datasets, decoding tokens one by one at sluggish speeds. Non-autoregressive approaches speed things up dramatically but often sacrifice linguistic accuracy, producing garbled or mispronounced output. A new approach from researchers Wasim Madha, Nityanand Mathur, Hamees Sayed, Apoorv Singh, Sameer Khurana, Akshat Mandloi, and Sudarshan Kamath, detailed in arXiv:2608.00011, aims to get the best of both worlds with DLLM-TTS.

DLLM-TTS formulates speech synthesis as conditional block discrete diffusion over X-Codec2 neural audio codec tokens. Instead of generating all tokens autoregressively or all at once, it splits sequences into blocks and applies masked diffusion within each block, while processing the blocks sequentially. This hybrid structure lets the model learn local acoustic coherence and global text-speech alignment simultaneously, capturing the reliability of autoregressive systems without losing the efficiency of parallel generation. During inference, parallel token prediction inside blocks delivers a real-time factor of just 0.15, meaning it generates speech about 6.7x faster than playback. Remarkably, a 0.6B-parameter model trained on only 20K hours of speech achieves competitive results on the Seed-TTS-eval benchmark, showing that block discrete diffusion enables practical, data-efficient, and fast TTS without the need for enormous training runs or huge model sizes.