The authors speed up inference in diffusion language models (dLLMs) by identifying tokens that converge early in the denoising process but don't yet meet the usual confidence thresholds. Their LEAP method uses lookahead filtering (checking future denoising steps) and multi-sequence superposition to detect these early-converging tokens and decode them in parallel without waiting for high confidence. This reduces denoising steps by ~30% on average and achieves 7.2 tokens per step on GSM8K while preserving accuracy.
Main takeaways:
- Existing dLLM parallelism relies on high-confidence thresholds, which are overly conservative: many tokens converge correctly early but don't reach the threshold.
- LEAP detects early-converging tokens by looking ahead at future denoising steps and checking alignment across multiple sequences.
- Training-free, plug-and-play method that reduces average denoising steps by ~30% compared to confidence-based decoding.
- On GSM8K, combines with dParallel to decode 7.2 tokens per step while preserving model precision.
- Breaks the reliance on high-confidence priors, offering a new paradigm for parallel decoding in diffusion language models.