Data-Forcing Distillation restores diversity and fidelity in few-step video gen

Recent few-step video generation models distilled via Distribution Matching Distillation (DMD) suffer from two critical failure modes: a sharp drop in sample diversity (mode collapse) and over-saturated outputs that look unnatural. A team of researchers from multiple institutions introduces Data-Forcing Distillation (DFD), a simple post-training framework that fixes both issues with minimal code changes. At its core, DFD leverages a teacher score discrepancy signal that pulls the student model toward real-data distribution modes it has missed while pushing it away from problematic over-saturated areas. This single-line modification to the DMD pipeline restores the generative fidelity and diversity that are essential for practical video generation.

DFD requires only 100–300 additional fine-tuning steps and has been validated on two large video generation models: Wan2.1-1.3B and Cosmos-Predict2.5-2B. The method not only eliminates over-saturation artifacts but also improves video dynamics and appearance to the point of outperforming the original teacher model. The paper provides theoretical analysis grounding why the teacher score discrepancy works, and demonstrates results across text-to-video, image-to-video, and autoregressive video generation tasks. For developers and researchers working on efficient video generation, DFD offers a practical, low-cost path to high-quality outputs without the trade-offs inherent in standard distillation.