EchoRL: Reinforcement Learning via Rollout Echoing

EchoRL introduces a method to address advantage degeneration in reinforcement learning with verifiable rewards (RLVR) for large language models. During post-training, a growing fraction of prompts produce rollouts where all self-generated responses are verified as successful, leading to zero standard deviation in rewards and zero advantage. This causes policy gradients to vanish, capping performance. EchoRL recovers learning signals from these degenerated rollouts by analyzing entropy patterns in golden trajectories from external expert models.