MCU: A New Approach to Continual Unlearning in Multimodal LLMs
A new approach called Merging for Continual Unlearning (MCU) has been developed by researchers to tackle the difficulties associated with continual unlearning in multimodal large language models (MLLMs). Conventional unlearning methods are often one-shot and struggle when faced with sequential unlearning requests. This repeated use of one-shot techniques can result in cumulative utility loss, unlearning rebound, and retention drift. MCU addresses this by merging multiple one-shot unlearning adapters into a single adapter with each incoming request. The research team found through leave-one-out merging analysis that these adapters show significant cross-task dependencies, which can enhance cross-task unlearning transferability but may also cause interference that reduces effectiveness. The study can be found on arXiv with the identifier 2608.04548.
Key facts
- MCU stands for Merging for Continual Unlearning.
- It addresses continual unlearning in multimodal large language models (MLLMs).
- Existing MLLM unlearning methods are one-shot and fail in continual scenarios.
- Repeated one-shot operations cause cumulative utility degradation, unlearning rebound, and retention drift.
- MCU dynamically merges one-shot unlearning adapters into a unified adapter.
- Leave-one-out merging analysis reveals cross-task dependencies among adapters.
- These dependencies have contrasting effects: transferability and interference.
- The paper is available on arXiv with ID 2608.04548.
Entities
Institutions
- arXiv