Prompt Codebooks: Discrete Optimization for LLM Instruction Refinement

A new framework called Prompt Codebooks (PCO) proposes a compositional approach to automatic prompt optimization (APO) for large language models. Unlike existing methods that treat prompts as monolithic strings, PCO decomposes prompts into atomic, reusable instruction units called instincts, organized in a discrete codebook. An LLM-based encoder routes each input to a subset of codebook entries, which a generator composes into a prompt for a frozen target model. A critic then provides structured feedback decomposed into per-variable textual gradients, enabling joint training of encoder, generator, and codebook under a language-valued min-max objective. This approach aims to produce more robust updates and allow reuse of learned sub-behaviors across tasks.