d3LLM: Pseudo-Distilled Diffusion LLM Balances Accuracy and Parallelism

A recent study published on arXiv (2601.07568) presents d3LLM (Pseudo-Distilled Diffusion Large Language Model), a novel approach aimed at addressing the accuracy-parallelism dilemma found in diffusion large language models (dLLMs). While dLLMs provide benefits like parallel decoding and random-order generation compared to autoregressive (AR) LLMs, implementing these advantages is often problematic. Traditional methods usually prioritize either efficiency or accuracy, but d3LLM innovatively combines both by utilizing pseudo-trajectory distillation in training, guiding the model on which tokens to decode confidently in early stages to enhance parallelism. For inference, it incorporates entropy-based multi-block decoding along with a KV-cache refresh mechanism to ensure high parallelism without sacrificing accuracy. Additionally, the paper introduces AUP (Accuracy under Parallelism) as a new metric for evaluating dLLMs. This research was categorized as a replace-cross type on arXiv.