δ-mem: Efficient Online Memory for Large Language Models
A novel lightweight memory system named δ-mem enhances frozen full-attention LLM backbones by integrating a compact online associative memory state. This mechanism condenses historical data into a fixed-size state matrix, which is updated through delta-rule learning, resulting in low-rank adjustments to attention calculations. Utilizing just an 8×8 state, δ-mem achieves an average score of 1.10× compared to the frozen backbone and 1.15× against the leading non-δ-mem baseline. In memory-intensive evaluations, improvements reach 1.31× on MemoryAgentBench and 1.20× on LoCoMo. This approach effectively tackles the inefficiencies and costs associated with enlarging context windows for long-term assistant and agent systems.
Key facts
- δ-mem is a lightweight memory mechanism for LLMs.
- It augments a frozen full-attention backbone with a compact online associative memory state.
- Past information is compressed into a fixed-size state matrix updated by delta-rule learning.
- Readout generates low-rank corrections to attention computation during generation.
- Uses only an 8×8 online memory state.
- Improves average score to 1.10× that of the frozen backbone.
- Outperforms strongest non-δ-mem memory baseline by 1.15×.
- Achieves 1.31× on MemoryAgentBench and 1.20× on LoCoMo.
Entities
Institutions
- arXiv