REFLEX: Refinement-Aware Expert Allocation for Diffusion Language Models
A new arXiv preprint (2608.01784) introduces REFLEX, a training-free method to improve Mixture-of-Experts (MoE) inference in diffusion language models (DLMs). The paper argues that MoE inference in DLMs should be viewed as refinement-aware compute allocation, as each denoising forward revisits all token positions with varying refinement demands, while default fixed token-choice routing assigns a uniform expert budget. REFLEX keeps the default router unchanged but reorganizes expert computation around evolving refinement states, aiming to better match expert computation with refinement needs. The method is proposed as a way to enhance efficiency in DLMs without additional training.
Key facts
- Preprint arXiv:2608.01784 introduces REFLEX.
- REFLEX is a training-free method for MoE inference in diffusion language models.
- It addresses the mismatch between uniform expert budget and varying refinement demands.
- The method keeps the default router unchanged.
- It reorganizes expert computation around evolving refinement states.
- The paper is published on arXiv.
- The announcement type is new.
- The method is proposed for diffusion language models.
Entities
Institutions
- arXiv