Cross-Model Memory Transfer via Target-Side Reader Adaptation
The preprint on ArXiv, numbered 2608.17050, explores the role of memory in large language models, particularly examining the effects of fixed memory content in contrast to the target-side reader during the transfer of a hashed-memory system. It identifies three types of knowledge enhancement: non-parametric retrieval, which allows for adaptable access but suffers from latency and limited integration; parametric adaptation, which embeds knowledge into model weights for better efficiency but complicates updates; and Engram-style hashed memory, which merges both approaches by keeping data in an external table alongside a trained reader. The research emphasizes cross-model applicability, analyzing how memory and reader contributions affect performance, with the goal of guiding the development of modular AI systems that are customizable and updatable without complete retraining.
Key facts
- ArXiv preprint 2608.17050 investigates cross-model memory transfer in large language models.
- Three knowledge-use regimes are identified: non-parametric retrieval, parametric adaptation, and Engram-style hashed memory.
- Non-parametric retrieval adds retrieval latency, context overhead, and offers only shallow integration.
- Parametric adaptation is efficient at inference but entangles knowledge with model weights.
- Engram-style hashed memory stores learned information in an external, addressable table.
- The central question is whether the frozen memory or the target-side reader matters more when moving memory across backbones.
- The method is cross-model frozen-memory extraction, training only a lightweight reader on the target model.
- The abstract mentions ablations but cuts off before presenting specific results.
Entities
Institutions
- arXiv