New adaptive scheduler for diffusion LLMs boosts multi-attribute control by 15%

Training large language models to generate controlled text often relies on uniform intervention at every denoising step, a method imported from autoregressive models. New research shows this uniform schedule actively degrades output quality, especially when steering multiple attributes simultaneously. To diagnose the issue, the team trained sparse autoencoders on four discrete diffusion language models ranging from 124M to 8B parameters. They discovered that different attributes commit on distinct schedules: topic solidifies within the first 2% of the denoising process, while sentiment emerges gradually over 20%. Uniform intervention wastes steering capacity on steps where the target attribute has already formed or hasn’t yet begun.

The proposed adaptive scheduler concentrates interventions only on steps where an attribute is actively forming. The advantage over uniform scheduling is captured by a single dispersion statistic of the commitment distribution. Tested across four DLMs and seven steering tasks, the adaptive method achieves precise control without degradation. In the challenging case of simultaneous three-attribute steering, it reaches 93% steering strength—outperforming the strongest baseline by up to 15 percentage points—while preserving generation quality. This work offers a principled, computationally efficient approach to controllable diffusion text generation.