Group Fine-Tuning (GFT): A Unified Post-Training Framework for LLMs

A recent study published on arXiv introduces Group Fine-Tuning (GFT), a comprehensive post-training framework designed for large language models that overcomes the challenges associated with supervised fine-tuning (SFT) and reinforcement learning (RL). The researchers investigate the dynamics of training and conclude that SFT represents a specific instance of policy gradient optimization characterized by sparse implicit rewards and unstable inverse-probability weighting. This leads to issues such as single-path dependency, entropy collapse, and gradient explosion. GFT features Group Advantage Learning, which creates varied response groups and employs normalized contrastive supervision to mitigate reward sparsity, along with Dynamic Coefficient Rectification, which adaptively regulates inverse-probability weights for stable training. The study can be found at arXiv:2604.14258.