GROM: One-Shot Machine Unlearning via Closed-Form Update
A recent paper published on arXiv (2608.05783) presents GROM, a novel gradient-free technique for swift one-shot machine unlearning in large language models. In contrast to existing leading methods that depend on iterative fine-tuning, GROM opts for a straightforward analytical solution without iterative optimization. It conceptualizes unlearning as a ridge-regularized least-squares issue, producing a closed-form additive update for specific weight matrices. This strategy tackles the high computational costs and the absence of clear analytical expressions in gradient-based approaches, which often only obscure targeted knowledge instead of eliminating it. GROM seeks to address the challenge of restoring supposedly erased knowledge when quantizing an unlearned model. This cross-type submission can be accessed at https://arxiv.org/abs/2608.05783, offering a safer way to eliminate sensitive information from LLMs, crucial for privacy and compliance. The authors advocate for GROM as a superior and more efficient alternative to current unlearning methods, potentially influencing AI safety and model governance.
Key facts
- GROM is a gradient-free, one-shot machine unlearning method.
- It uses a closed-form additive update for targeted weight matrices.
- The method frames unlearning as a ridge-regularized least-squares optimization problem.
- It abandons iterative optimization used in current state-of-the-art approaches.
- Existing gradient-based methods are computationally expensive and lack explicit analytical formulations.
- Quantizing an unlearned model can restore much of the supposedly erased knowledge.
- The paper is available on arXiv with identifier 2608.05783.
- The method targets safe removal of specific, sensitive knowledge from large language models.
Entities
Institutions
- arXiv