Dataless Task Vector Disentanglement via Kronecker-Factored Curvature
A new method from arXiv (2602.17385) addresses cross-task interference in Task Arithmetic for foundation models. Task Arithmetic allows modular adaptation by combining task vectors, but this often causes representation drift and performance loss. Existing regularization techniques require external task data, violating modularity and privacy constraints. The proposed approach reframes drift regularization as a curvature matrix approximation problem, using Kronecker-Factored Approximate Curvature (K-FAC) to create a dataless regularizer. It achieves state-of-the-art results in task addition and negation, with constant complexity in the number of tasks and robustness to task vector rescaling, eliminating the need for external data.
Key facts
- arXiv paper 2602.17385 proposes dataless weight disentanglement for Task Arithmetic.
- Task Arithmetic combines task vectors for modular foundation model adaptation.
- Cross-task interference causes representation drift and degraded performance.
- Existing regularization requires external task data, conflicting with modularity and privacy.
- New method uses Kronecker-Factored Approximate Curvature (K-FAC) as a regularizer.
- Achieves state-of-the-art results in task addition and negation.
- Method has constant complexity in the number of tasks.
- Promotes robustness to task vector rescaling, eliminating need for external data.
Entities
Institutions
- arXiv