RIFT: A New Framework for LLM Alignment Using Negative Samples

Researchers propose Reward Informed Fine-Tuning (RIFT), a framework for aligning large language models (LLMs) that repurposes negative samples instead of discarding them. Unlike Supervised Fine-Tuning (SFT), which relies on costly expert data, and Rejection Sampling Fine-Tuning (RFT), which uses hard thresholding to discard negative trajectories, RIFT reweights the loss with scalar rewards to learn from both positive and negative self-generated samples. A stabilized loss formulation prevents training collapse caused by naive reward integration. Experiments on mathematical benchmarks across various base models show RIFT consistently outperforms RFT. The paper is available on arXiv.