Evolution Strategies Boost LLM Solution Coverage Beyond RL

A recent study published on arXiv (2608.12679) suggests leveraging Evolution Strategies (ES) to enhance the solution coverage of Large Language Models (LLMs) in fields such as mathematics and science. While the conventional method, pass@k, enables models to produce a variety of candidate solutions, applying Reinforcement Learning (RL) post-training tends to concentrate the output distribution on high-reward solutions, leading to diminished coverage. In contrast, ES is a gradient-free, population-based approach that directly optimizes in weight space through random changes, consistently outperforming RL in pass@k and yielding a wider output distribution with increased solution diversity. The paper emphasizes ES as a promising substitute for RL in post-training LLMs, thereby boosting their exploratory abilities.