LLM-Guided Bayesian Optimization Framework for Scientific Discovery

A novel approach known as LLM-Guided Bayesian Optimization (LGBO) incorporates large language models into the Bayesian optimization process, enhancing the efficiency of scientific exploration. This framework features a region-lifted preference mechanism, which integrates LLM-derived preferences into each iteration, allowing for a stable and manageable adjustment of the surrogate mean. It effectively tackles the issues of slow cold-start performance and limited scalability in high-dimensional contexts that hinder conventional Bayesian optimization. Unlike earlier methods that utilized LLMs solely for warm-start initialization or generating candidates, LGBO is the first to seamlessly weave semantic reasoning from LLMs into the optimization cycle. Theoretical evidence supports the claim that LGBO delivers superior performance.