Knowledge Capsules: Structured Memory Units for LLMs

Researchers have introduced Knowledge Capsules, which are nonparametric memory units designed to encapsulate normalized relational knowledge derived from document collections using a static base model. Rather than embedding knowledge as text, they present an External Key Value Injection (KVI) framework that transforms capsules into key value representations compatible with attention mechanisms. This innovation allows external knowledge to be integrated directly into the model's attention calculations. It effectively resolves the challenges faced by retrieval-augmented generation (RAG), where external knowledge competes as tokens, resulting in indirect and often unreliable effects, especially in scenarios involving lengthy contexts and multi-hop reasoning. This technique enables the updating or expansion of LLM knowledge without necessitating retraining, thus eliminating the expenses associated with altering parametric weights.