FoCore: Self-Contrast Decoding for Diffusion LLMs

A new decoding strategy called Focus on the Core (FoCore) has been proposed for Diffusion Large Language Models (DLMs). The method, detailed in arXiv preprint 2605.01373, addresses the limitation of current decoding strategies that exhibit local preference and overlook heterogeneous information density. Researchers identified that high-information-density (HD) tokens, which converge earlier than surrounding tokens, significantly improve output quality when explicitly conditioned. FoCore is a training-free approach that uses HD tokens in a self-contrast manner by temporarily remasking them as negative samples to guide generation. An accelerated variant, FoCore_Accel, is also introduced. The work highlights the distinct advantage of DLMs in global context modeling through iterative denoising.