Wasserstein Gradient Flow Improves Discrete Image Tokenizer Training

A recent paper on arXiv (2605.06148) presents a novel approach for aligning discrete image tokenizers with autoregressive (AR) priors throughout the training process. The conventional two-stage training first focuses on reconstruction, followed by fitting a prior model to static token sequences. This often leads to a discrepancy, as tokens retain image details but are challenging for AR models to predict. The authors investigate this issue using Tripartite Variational Consistency (TVC), which breaks down latent-variable learning into three essential conditions: conditional-likelihood, prior, and posterior consistency. They demonstrate that two-stage training only guarantees the first condition, neglecting prior consistency. To address this, they introduce a distribution-level prior-matching signal via Wasserstein gradient flow during tokenizer training, allowing the tokenizer to produce tokens that are both reconstructive and easily predictable. Their method is tested on standard image generation benchmarks, indicating enhanced AR prior fitting and improved generation quality.