LLM Harness Design Boosts Algorithm Discovery in Vesper Framework

A new arXiv preprint investigates how execution infrastructure design, or 'harness,' impacts automated algorithm discovery using large language models (LLMs) and evolutionary search. Building on AlphaEvolve and FunSearch, the study poses three key questions: whether to generate many algorithms with brief reasoning or fewer with deeper reasoning under a fixed token budget; how to handle evaluation hacks where programs exploit scoring functions; and how to safely parallelize agents requiring full filesystem access. The authors present Vesper, a framework incorporating harness improvements that address these issues. Evaluated on Circle Packing under identical token budgets, Vesper shows that generating fewer algorithms with deeper thought yields better results. The work emphasizes that discovery success depends not only on model capability but significantly on harness design.