Domino: New speculative decoding method speeds up Qwen3 by 5.8x
Decoupling causal modeling from drafting yields massive throughput gains on Qwen3 models.
Deep Dive
Up to 5.8x throughput speedup on Qwen3. Paper: https://arxiv.org/abs/2605.29707 Code: https://github.com/jianuo-huang/Domino Models: https://huggingface.co/Huang2020
Key Points
- Domino decouples causal modeling from autoregressive drafting, enabling longer parallel token generation.
- Achieves up to 5.8x throughput speedup on Qwen3 models with no quality loss.
- Open-source code and pre-trained models available on GitHub and Hugging Face for easy adoption.
Why It Matters
Faster LLM inference without quality loss means lower serving costs and better user experience in production.