Markov Chain Decoders Overcome Heavy-Tail Limits of Lipschitz Generative Models
A recent paper on arXiv (2605.18931) highlights a significant drawback of conventional deep generative models: their failure to generate heavy-tailed outputs, which frequently appear in areas like performance assessment, network traffic, and risk analysis. The researchers demonstrate that Variational Autoencoders (VAEs) utilizing Gaussian decoder likelihoods and Lipschitz-constrained neural networks are incapable of producing heavy-tailed distributions, as the Gaussian tail diminishes exponentially and Lipschitz continuity restricts the amplification of rare occurrences. They offer both theoretical insights and empirical evidence using synthetic Pareto data for tail indices α ∈ {2, 3, 5, 30} and dimensions d ∈ {1, 5, 10}. To address this issue, they substitute the Gaussian decoder with a Phase-Type (PH) distribution derived from Markov chains, leaving the encoder and latent space intact.
Key facts
- Paper arXiv:2605.18931 identifies heavy-tail limitation in Lipschitz generative models
- Standard VAEs with Gaussian decoders cannot produce heavy-tailed outputs
- Lipschitz continuity prevents amplification of rare events from latent space
- Empirical validation uses synthetic Pareto data with tail indices 2, 3, 5, 30
- Dimensions tested: 1, 5, 10
- Proposed solution replaces Gaussian decoder with Phase-Type (PH) distribution based on Markov chains
- Encoder and latent space remain unchanged in the proposed method
- Heavy-tailed distributions are prevalent in performance evaluation, network traffic, and risk modeling
Entities
—