OPDLM converts GPT-style models to diffusion with 7,000x fewer tokens

A team of researchers from Texas A&M University and other institutions have published a new paper introducing the On-Policy Diffusion Language Model (OPDLM). The work tackles the problem of converting pre-trained autoregressive language models (ARLMs), like GPT, into diffusion language models (DLMs) without the extreme computational costs typically required for training DLMs from scratch. The core innovation is a technique called On-Policy Distillation (OPD). Most prior ARLM-to-DLM conversion methods suffer from two distribution shifts: first, switching from next-token prediction to a diffusion objective discards learned knowledge; second, standard DLMs train on random masks but decode via confidence-based sampling, causing a train-inference mismatch.

OPDLM solves both problems by having a student model (bidirectional attention, derived from the ARLM architecture) generate its own inference-style trajectories during training. A frozen teacher model (the original ARLM) then provides target logits on those same trajectories. This on-policy approach eliminates the train-inference mismatch, while distillation preserves the teacher's knowledge. Empirically, OPDLM matches or exceeds prior DLM conversion methods using just 15x to 7,000x fewer training tokens. The method turns DLM conversion into a lightweight post-training step rather than full pretraining, making it dramatically more data-efficient and accessible.