New AI Algorithm MCPO Addresses Limitations in Reinforcement Learning for Large Language Models

A new research paper introduces Mastery-Consolidated Policy Optimization (MCPO), an algorithm designed to improve reinforcement learning for large language models. The work addresses specific issues found in existing Group Relative Policy Optimization (GRPO) approaches. When models achieve perfect accuracy on certain prompts, GRPO methods lose training signals, potentially causing the model to forget previously learned patterns. For prompts where the model is mostly correct but not perfect, the training signal weakens as accuracy increases, hindering progression to full mastery. MCPO incorporates a hinge-KL regularizer specifically for mastered prompts to prevent policy drift. It also employs a query weighting scheme that strengthens learning from partially correct responses. The research focuses on Reinforcement Learning with Verifiable Rewards (RLVR), a field aimed at enhancing the reasoning capabilities of LLMs. The paper was published on arXiv with the identifier 2604.16972v1.