Adaptive Entropy Regularization Framework Proposed to Enhance LLM Reasoning in Reinforcement Learning

A new research paper proposes Adaptive Entropy Regularization (AER) to address policy entropy collapse in Reinforcement Learning with Verifiable Rewards (RLVR) for Large Language Models. The work argues that entropy regularization's potential has been underestimated due to sensitivity to fixed coefficients. Analysis reveals that tasks of varying difficulty require different exploration intensities, and balanced exploration needs policy entropy maintained within a moderate range below initial levels. RLVR has emerged as a key paradigm to enhance reasoning ability in LLMs, but training often suffers from overly deterministic policies that hinder exploration. The framework dynamically adjusts regularization to unlock more stable performance across diverse tasks and models. This approach aims to improve reasoning performance by preventing entropy collapse during training. The research was published on arXiv with identifier arXiv:2510.10959v3.