SMCEvolve: Principled Scientific Discovery via Sequential Monte Carlo Evolution
The recently introduced framework, SMCEvolve, reinterprets the evolution of LLM-driven programs as sampling from a target distribution that is tilted by rewards, using a Sequential Monte Carlo (SMC) sampler for approximation. This method features three primary components: adaptive resampling of parents, a combination of mutation with acceptance, and control over automatic convergence. An analysis of finite-sample complexity establishes limits on the LLM-call budget for achieving a specific target error. In various benchmarks, including mathematics, algorithm efficiency, symbolic regression, and comprehensive ML research, SMCEvolve outperforms leading evolving systems while utilizing fewer LLM calls with self-determined termination. The code is publicly accessible.
Key facts
- SMCEvolve recasts program search as sampling from a reward-tilted target distribution.
- It uses a Sequential Monte Carlo (SMC) sampler for approximation.
- Three core mechanisms: adaptive parent resampling, mixture of mutation with acceptance, automatic convergence control.
- Provides finite-sample complexity analysis bounding LLM-call budget.
- Tested on math, algorithm efficiency, symbolic regression, and end-to-end ML research benchmarks.
- Outperforms state-of-the-art evolving systems with fewer LLM calls.
- Code is available.
- arXiv:2605.15308v1.
Entities
Institutions
- arXiv