DiffRetriever beats autoregressive retrieval with parallel token generation
Bidirectional diffusion retrieves K tokens in one pass—no sequential bottleneck.
A team from the University of Queensland (Shuai Wang, Yin Yu, Shengyao Zhuang, Bevan Koopman, Guido Zuccon) has released DiffRetriever, a new approach that exploits diffusion language models for dense and sparse retrieval. The key insight: previous attempts to generate multiple representative tokens for queries and passages failed to improve over single-token decoding because autoregressive generation is inherently sequential. DiffRetriever sidesteps this by appending K masked positions to the prompt and reading all K in a single bidirectional pass—the same latency regardless of K.
In both in-domain and out-of-domain evaluations on BEIR-7, multi-token DiffRetriever consistently outperformed single-token baselines across multiple diffusion backbones, while autoregressive multi-token showed flat or negative gains with linearly increasing latency. After supervised fine-tuning on the Dream backbone, DiffRetriever became the strongest retriever among all compared systems, beating PromptReps, encoder-style DiffEmbed, and the contrastively fine-tuned RepLLaMA. The authors also show that an oracle selecting different numbers of tokens per query on the frozen base model exceeds contrastive fine-tuning at the same fixed budget, pointing to adaptive budget selection as future work. Code is available on GitHub.
- DiffRetriever uses K masked tokens decoded in one bidirectional pass, eliminating the sequential latency of autoregressive models.
- On BEIR-7 after fine-tuning on Dream, it outperforms PromptReps, DiffEmbed, and contrastive RepLLaMA.
- A per-query oracle on the frozen model surpasses contrastive fine-tuning, suggesting future adaptive token budgets.
Why It Matters
Faster, more accurate retrieval without sequential bottlenecks—key for real-time search and RAG pipelines.