Rotation-Preserving Fine-Tuning Improves LLM Generalization
A novel technique known as Rotation-Preserving Supervised Fine-Tuning (RPSFT) enhances the balance between in-domain efficacy and out-of-domain adaptability in large language models. Introduced in arXiv:2605.10973, RPSFT imposes penalties on alterations within the projected top-k singular-vector block of pretrained weight matrices. This approach serves as an effective substitute for Fisher-sensitive directions, avoiding the high computational demands associated with Hessian or Fisher information. By minimizing unnecessary rotations while maintaining task adaptation, RPSFT surpasses standard SFT and competitive benchmarks across various model families and sizes trained on mathematical reasoning data, effectively preserving pretrained representations and improving the in-domain/out-of-domain trade-off.
Key facts
- RPSFT stands for Rotation-Preserving Supervised Fine-Tuning
- Method proposed in arXiv:2605.10973
- RPSFT penalizes changes in projected top-k singular-vector block of pretrained weight matrices
- It is an efficient proxy for Fisher-sensitive directions
- Avoids computationally expensive Hessian or Fisher information at LLM scale
- Tested across model families and sizes on math reasoning data
- Improves in-domain/OOD trade-off over standard SFT and strong baselines
- Better preserves pretrained representations
Entities
—