Google DeepMind's Brendan O’Donoghue demystifies text diffusion models
New talk explains DiffusionGemma's core technology just after its release.
Deep Dive
A video released just a week ago, right before the DiffusionGemma paper, answers key questions and clears up confusion about the release. It's even more relevant now.
Key Points
- Brendan O’Donoghue explains how diffusion applies to discrete text tokens, not just continuous data.
- The talk clarifies confusion around DiffusionGemma's training and sampling, including score matching objectives.
- Highlights advantages of text diffusion: parallel generation, controllable outputs, and improved diversity.
Why It Matters
This talk equips professionals with the intuition to build with or improve upon the next generation of non-autoregressive language models.