GM-RBM: A Potts Model Extension of the Gaussian-Bernoulli RBM
A new generative energy-based model, the Gaussian-Multinoulli Restricted Boltzmann Machine (GM-RBM), has been developed by researchers. This model enhances the Gaussian-Bernoulli RBM (GB-RBM) by substituting binary hidden units with q-state categorical (Potts) units, thereby offering a more complex latent state space suitable for multivalued concepts. Such advancements are advantageous for tasks demanding discrete, structured representations like associative memory and symbolic reasoning. The paper, accessible on arXiv (2505.11635), includes a comprehensive derivation of the energy function, learning rules, and conditional distributions. It also discusses training methodologies, such as contrastive divergence with temperature annealing. The authors assess the GM-RBM in both capacity-matched and parameter-matched scenarios to distinguish architectural effects from latent capacity limitations.
Key facts
- The GM-RBM extends the Gaussian-Bernoulli RBM by using q-state categorical (Potts) units instead of binary hidden units.
- The model is designed to handle multivalued concepts and discrete, structured representations.
- The paper provides a self-contained derivation of the energy, conditional distributions, and learning rules.
- Training choices include contrastive divergence with temperature annealing and intra-slot diversity constraints.
- Evaluation is done under capacity-matched and parameter-matched setups against GB-RBM.
- The paper is available on arXiv with identifier 2505.11635.
- The announcement type is 'cross', indicating a revision.
- The work aims to improve associative memory and symbolic reasoning tasks.
Entities
Institutions
- arXiv