Energy-Navigated Distillation Improves Few-Step Discrete Flow Matching
A new method called Trajectory-Shaped Discrete Flow Matching (TS-DFM) improves few-step text generation by replacing blind stochastic jumps in training trajectories with guided navigation using a lightweight energy compass. The approach addresses the bottleneck of poor-quality trajectories rather than insufficient student capacity. TS-DFM evaluates candidate continuations at each midpoint during training, selecting the most coherent path, while inference cost remains unchanged. Experiments on 170M-parameter language modeling demonstrate effectiveness. The paper is published on arXiv under ID 2605.07924.
Key facts
- TS-DFM replaces blind stochastic jumps with guided navigation in training trajectories.
- A lightweight energy compass evaluates candidate continuations at each midpoint.
- All shaping is training-only; inference cost is unchanged.
- Method addresses trajectory bottleneck, not student capacity.
- Experiments on 170M-parameter language modeling.
- Paper published on arXiv with ID 2605.07924.
- Discrete flow matching generates text by iteratively transforming noise tokens.
- Distillation uses multi-step trajectory to train a student for few-step generation.
Entities
Institutions
- arXiv