Continuous Optimization via Latent Heuristic Search for Automated Algorithm Design

A new framework from arXiv (2605.17137) proposes continuous heuristic discovery for automated algorithm design, shifting optimization from discrete program syntax to a learned latent manifold. The method uses an encoder to map discrete programs into continuous embeddings, trains a differentiable surrogate model to predict performance, and employs gradient-based search. An invertible normalizing flow regularizes the optimization by mapping embeddings to a structured Gaussian prior, where gradient ascent is performed. Optimized latent vectors are projected via a learned mapper into soft prompts, conditioning a frozen Large Language Model (LLM) to synthesize novel executable heuristics. This approach addresses the non-convex optimization landscape of traditional evolutionary methods that rely on stochastic sampling.