Diffusion LMs set new record in lossless text compression speed
First diffusion-LM compression framework beats autoregressive LLMs on enwik8 throughput.
Nardone and Ferragina's new paper, Diffuse to Compress: Leveraging Diffusion LMs for Lossless Compression (arXiv:2608.11249), applies Diffusion Language Models to lossless text compression for the first time. Traditional LLM-based compressors, while beating general-purpose tools like zstd and gzip on text and code, are bottlenecked by autoregressive decoding: they generate one symbol per forward pass, making them too slow for practical use. Diffusion LMs break this constraint by deciding the number and positions of symbols encoded at each turn, enabling compression in fewer, parallel steps.
The authors solve key algorithmic challenges unique to diffusion-based compression and validate their framework experimentally on enwik8, a standard compression benchmark. Their DLM-based system advances the state of the art in lossless text compression while sidestepping the throughput limits of autoregressive methods. Because DLMs are still evolving rapidly, the team expects further gains as model efficiency improves. The 18-page paper includes 11 figures and 2 tables of comparisons, positioning diffusion models as a serious alternative for practical neural compression.
- First-ever use of Diffusion Language Models (DLMs) for lossless text compression, replacing autoregressive LLMs.
- Overcomes the one-symbol-per-step throughput bottleneck by encoding multiple symbols per forward pass.
- Achieves state-of-the-art compression on the enwik8 benchmark, surpassing LLM-based and zstd/gzip/bzip2 compressors.
Why It Matters
Faster lossless compression could make LLM-based text compression practical for storage-heavy industries.