Google DeepMind's DiffusionGemma generates text 1000+ tokens/sec via diffusion
Non-autoregressive model processes 256-token blocks at once on a single H100.
Google DeepMind has introduced DiffusionGemma, an experimental open-weights model that fundamentally rethinks how text generation works. Released under the Apache 2.0 license, it replaces the standard autoregressive token-by-token approach with a text diffusion head. The process starts by placing a 256-token 'canvas' of random placeholder noise, then uses Uniform State Diffusion to iteratively refine and denoise the entire block simultaneously. Because every token can attend to every other token (bidirectional context), highly confident tokens help snap adjacent tokens into focus over multiple passes. An innovative Error Correction via Re-Noising mechanism allows the model to introduce noise mid-generation if its confidence drops, enabling real-time self-correction.
The performance impact is dramatic. By processing entire blocks at once, DiffusionGemma shifts the inference bottleneck from memory bandwidth to raw compute, achieving over 1,000 tokens per second on a single NVIDIA H100 and over 700 tokens per second on a local RTX 5090. The model is built on Gemma 4 architecture as a 26-billion-parameter Mixture of Experts (MoE), but it activates only 3.8 billion parameters during inference. When quantized, it comfortably fits within an 18GB VRAM footprint, making it highly accessible for local PC workflows. This breakthrough could unlock faster, more interactive AI text generation for developers and researchers running models on consumer hardware.
- Text generation via diffusion instead of autoregressive, processing 256-token blocks in parallel.
- 26B MoE model with only 3.8B active parameters, hitting 1000+ tokens/sec on H100 and 700+ on RTX 5090.
- Fits in 18GB VRAM when quantized, enabling local deployment on consumer GPUs with real-time error correction.
Why It Matters
Opens door to massively faster, local text generation with non-autoregressive architectures, challenging the token-by-token paradigm.