CForce: Consistency Forcing Boosts Parallel Decoding for Diffusion LLMs

Researchers have introduced Consistency Forcing (CForce), a distillation method designed to improve the reliability of diffusion large language models (dLLMs) during parallel decoding. dLLMs generate text by predicting multiple masks in a single forward pass, but aggressive parallelism can lead to unreliable predictions in early denoising stages, causing errors that propagate. CForce addresses this by forcing early-stage mask predictions to align with later-stage predictions, training the model on pre-collected self-rollout trajectories to improve training-inference alignment. The method employs Confidence Adaptive KL Divergence as a distillation objective, combining the strengths of forward and reverse KL divergence. Theoretical analysis shows that CForce approximately minimizes prediction error in early stages. The approach is applicable to both mask-to-token and other decoding paradigms. The paper is available on arXiv under the identifier 2608.13925.