ABC-GRPO: Bounded Clipping for Stable LLM Reinforcement Learning

A new reinforcement learning algorithm, All-Quadrant Bounded Clipping GRPO (ABC-GRPO), has been proposed to address a critical flaw in Group Relative Policy Optimization (GRPO), a popular method for training large language models (LLMs). The research, detailed in arXiv paper 2601.03895, identifies that GRPO, which inherits PPO's token-level clipping but uses sequence-level advantages, leaves one quadrant of the (likelihood-ratio, advantage) space structurally unbounded. Specifically, in the case of negative advantage combined with an increased likelihood ratio (Q4), a few high-ratio tokens can receive very large suppressive updates, leading to collapsed entropy and narrowed reasoning boundaries. ABC-GRPO applies unconditional clipping in all four quadrants using sign-dependent boundaries, clipping the likelihood ratio before multiplication by the advantage. This adds a trust-region floor in Q2 and a cap in Q4, ensuring stable and generalizable training. The paper is available on arXiv and was announced as a replace-cross type update.