Predict-then-Diffuse: Adaptive Response Length for Compute-Budgeted Inference in Diffusion LLMs

A new framework called Predict-then-Diffuse addresses the fixed-size response length limitation in Diffusion-based Large Language Models (D-LLMs). These models generate tokens in parallel, offering throughput advantages over autoregressive models, but require a predetermined response length. This leads to computational waste on padding tokens if oversized, or truncation and costly re-computations if undersized. The proposed method estimates the appropriate response length per query before inference, enabling compute-budgeted generation. It is model-agnostic and aims to optimize GPU utilization and reduce latency spikes. The paper is available on arXiv with ID 2605.04215.