DLMRec: Diffusion language model reimagines LLM-powered recommendations
New discrete diffusion beats autoregressive LLMs for modeling user preferences and item dependencies.
Large language models (LLMs) have been increasingly used to power recommender systems by leveraging semantic reasoning and generative capacity. However, most existing approaches rely on autoregressive generation, which has two key drawbacks for recommendation: the next-token prediction objective prioritizes sequential order over the structural dependencies between items that actually drive user preferences, and the prefix-constrained left-to-right decoding prevents bidirectional context, causing early errors to compound without correction.
To address these limitations, the team proposes DLMRec (Diffusion Language Model for Recommendation), a discrete diffusion language model that treats recommendation as a denoising process over item sequences. The model introduces three core components: a collaborative-aware stochastic tokenizer that encodes multi-hop collaborative signals into discrete tokens compatible with diffusion; a curriculum-driven training strategy that progressively learns item- and token-level preference recovery; and a stability-aware voting mechanism that aggregates iterative predictions for more robust generation. This approach avoids the autoregressive pitfalls and better captures the complex, diverse patterns of user behavior. The paper (arXiv:2607.21519) includes 9 figures and 30 pages of technical detail.
- DLMRec uses discrete diffusion instead of autoregressive generation to avoid error accumulation from left-to-right decoding.
- Collaborative-aware stochastic tokenizer encodes multi-hop collaborative signals into discrete tokens for diffusion modeling.
- Curriculum-driven training progressively aligns denoising with preference recovery at both item and token levels.
Why It Matters
Could replace autoregressive LLMs in recommendation systems, improving accuracy and diversity without sequential decoding errors.