Learning to Denoise at the Right Time Slashes Training by 4x and Outperforms 1B-Parameter Models

Researchers Bingshuo Qian and Xiang Cheng from the paper "Learning When to Denoise" tackle a critical inefficiency in multi-representation diffusion models: the schedule that determines when each representation is denoised. Instead of using a fixed schedule, they propose learning it via a schedule-corrected objective and a flexible parametric class that is convex and monotone by construction. This allows the model to automatically optimize the order of denoising across complementary views of an image, requiring less than 1% additional training compute.

On ImageNet 256x256, the learned schedule delivers dramatic improvements. With AutoGuidance, a 200-epoch model achieves FID 1.05, matching the 800-epoch SFD-XL baseline — a 4x training speedup. Extending to 600 epochs yields FID 1.02, which beats the 1B-parameter SFD-XXL result of FID 1.04 while using a smaller 675M-parameter backbone. Even without guidance, the 200-epoch model reaches FID 2.37, below the best 800-epoch SFD-XL (2.54), and improves to FID 2.14 at 600 epochs. The code is publicly available, signaling a practical path to cheaper, faster, and higher-quality image synthesis.