CRAFT: A New Continual Learning Framework for LLMs
Researchers propose CRAFT, a continual learning framework for large language models that mitigates catastrophic forgetting by learning low-rank interventions on hidden representations instead of updating model weights. The method operates in three stages: routing tasks to similar groups based on output-distribution divergence, fine-tuning with KL divergence against the group's prior state to control forgetting, and merging interventions using the same KL signal. This unified approach improves performance and reduces forgetting compared to LoRA-based methods across multiple benchmarks and model scales.
Key facts
- CRAFT avoids updating model weights by learning low-rank interventions on hidden representations.
- The framework routes each task to a group of similar tasks based on output-distribution divergence.
- Fine-tuning uses KL divergence against the group's prior state to control forgetting and determine convergence.
- Interventions for updated tasks are merged into the shared representation using the same KL signal.
- CRAFT unifies routing, regularization, and merging through a single KL-based objective.
- CRAFT improves overall performance and reduces forgetting compared to strong LoRA-based approaches.
- The method is evaluated across multiple benchmarks and model scales.
- The paper is available on arXiv with ID 2605.05732.
Entities
—