MMGuard: Proactive Protection Against Unauthorized LVLM Fine-Tuning
Researchers propose MMGuard, a method to protect multimodal data from unauthorized fine-tuning of Large Vision-Language Models (LVLMs). Unlike post-hoc approaches like machine unlearning and watermarks, MMGuard proactively generates unlearnable examples by injecting human-imperceptible perturbations that exploit LVLM learning dynamics. These perturbations minimize training loss, creating an optimization shortcut that causes models to overfit to noise, degrading performance when perturbations are absent during inference. The approach addresses copyright and privacy risks from unauthorized scraping and training on multimodal web data.
Key facts
- MMGuard protects multimodal data from unauthorized LVLM fine-tuning.
- It generates unlearnable examples with human-imperceptible perturbations.
- Perturbations exploit LVLM learning dynamics by minimizing training loss.
- This creates an optimization shortcut, causing overfitting to noise.
- Performance degrades when perturbations are absent during inference.
- Existing countermeasures like machine unlearning and watermarks are post-hoc.
- Unauthorized scraping and training pose copyright and privacy risks.
- MMGuard is a proactive defense mechanism.
Entities
—