The authors study controlled text generation in discrete diffusion language models (DLMs), which generate text by denoising all positions in parallel rather than left-to-right. They find that applying steering interventions uniformly across all denoising steps degrades quality, especially when steering multiple attributes at once. Using sparse autoencoders trained on four DLMs, they discover that different attributes (topic, sentiment, etc.) "commit" at different points in the denoising schedule — topic solidifies in the first 2% of steps, while sentiment emerges gradually over 20%. They propose an adaptive scheduler that concentrates interventions only when each attribute is actively forming.
Main takeaways:
- Uniform steering at every denoising step in DLMs wastes effort on timesteps where the target attribute has already solidified or hasn't emerged yet, degrading generation quality.
- Different attributes commit on distinct schedules: topic commits early (first 2% of denoising), sentiment commits gradually (over 20%).
- An adaptive scheduler that intervenes only during attribute formation achieves up to 93% steering strength on three-attribute control, beating the best baseline by 15 points while preserving quality.
- Sparse autoencoders trained on DLMs reveal when and how strongly different attributes emerge during generation.
- The advantage of adaptive over uniform scheduling is governed by a single "dispersion statistic" of the commitment distribution, giving a closed-form cost-control trade-off.