Stochasticity Improves Discrete Diffusion Sampling Quality

A new study from arXiv (2605.26582) systematically examines how the degree of stochasticity in Markov transitions affects the tradeoff between sampling efficiency and sample quality in discrete diffusion models. The authors demonstrate that highly deterministic transitions converge quickly but accumulate errors, whereas more stochastic transitions converge slowly yet can achieve higher final sample quality. Through information-theoretic analysis, they identify an error-correcting mechanism driven by redundant transitions that symmetrically exchange mass between states, which provably contracts sampling errors. Based on this insight, they propose Discrete Churn and Restart Sampling (DCRS), a novel inference algorithm that injects controlled stochasticity to improve performance.