Parametric Memory Head for Continual Generative Retrieval
A new paper on arXiv (2604.23388) introduces post-adaptation memory tuning (PAMT) to address catastrophic forgetting in generative information retrieval (GenIR). GenIR uses a single neural model to decode document identifiers from queries, but struggles with dynamic collections. Standard fine-tuning degrades performance on earlier data. PAMT freezes the backbone and attaches a product-key memory head, improving stability-plasticity balance.
Key facts
- arXiv paper 2604.23388
- Proposes post-adaptation memory tuning (PAMT)
- Addresses catastrophic forgetting in GenIR
- Uses a parametric memory head (PMH)
- Freezes the backbone model
- Attaches a product-key memory
- Improves stability-plasticity trade-off
- Focuses on continual generative retrieval
Entities
Institutions
- arXiv