ArcVQ-VAE Introduces Spherical Angular-Margin Prior for Better Discrete Representations
A recent publication on arXiv presents ArcVQ-VAE, an advanced vector quantization framework that enhances the traditional VQ-VAE by incorporating a spherical angular-margin prior (SAMP) for its codebook. The SAMP features Ball-Bounded Norm Regularization, which keeps codebook vectors within a time-varying Euclidean ball, alongside ArcCosine Additive Margin Loss, which improves angular separability of latent vectors. This method seeks to address the capacity constraints of standard VQ-VAE models in effectively capturing complex and varied representations for image modeling. The research is documented under arXiv ID 2605.13517.
Key facts
- ArcVQ-VAE is a novel vector quantization framework.
- It introduces a spherical angular-margin prior (SAMP) for the codebook.
- SAMP includes Ball-Bounded Norm Regularization and ArcCosine Additive Margin Loss.
- Ball-Bounded Norm Regularization constrains codebook vectors within a time-dependent Euclidean ball.
- ArcCosine Additive Margin Loss encourages greater angular separability among latent vectors.
- The framework aims to improve effective latent-space utilization in VQ-VAE.
- The paper is available on arXiv with ID 2605.13517.
- The work addresses capacity limitations of conventional VQ-VAE models.
Entities
Institutions
- arXiv