CMP Architecture Reduces Catastrophic Forgetting in Continual Learning

A recent paper published on arXiv presents a novel continual-learning framework known as CMP (Cognitive Memory Primitive). This architecture encodes inputs as sparse relational codes and utilizes a two-tier competitive memory system, enabling learning through localized updates without the need for end-to-end backpropagation within its feature-generating framework. CMP is designed to mitigate the issue of catastrophic forgetting, which is common in traditional backpropagation-based continual-learning methods. In a controlled domain-incremental byte-level language modeling task, CMP showed significantly reduced backward transfer compared to a parameter-equivalent Transformer that employed online Elastic Weight Consolidation (EWC). In a replicated 15-domain experiment with three seeds, CMP maintained stable forgetting patterns, consistently outperforming the Transformer baselines in head-to-head and domain-order assessments. The paper can be found on arXiv with ID 2607.26523.