Safety-Aware Probing Framework Prevents LLM Safety Degradation During Fine-Tuning

A new study from arXiv (2505.16737) introduces a safety-aware probing (SAP) optimization framework to prevent large language models from losing their safety alignment during fine-tuning. The researchers demonstrate that safety and task-performance loss landscapes are partially decoupled, meaning updates that improve task-specific performance can inadvertently shift the model toward unsafe regions. SAP uses contrastive safety signals to identify safety-correlated directions and optimizes a lightweight probe to maintain safety constraints. The paper revisits the fundamental question of why fine-tuning on non-harmful data can degrade safety, offering a solution to preserve alignment without sacrificing task performance.