OPTI-Q: Database-Inspired Optimizer for Multi-LLM Question Planning

A new optimization framework called OPTI-Q has been developed by researchers for multi-LLM question planning, focusing on constraint-based optimization. This system represents LLM invocations as physical operators within an execution DAG, aiming to find plans that enhance answer quality while balancing costs, latency, and energy based on user-defined constraints. To estimate quality and resource costs without executing every candidate plan, OPTI-Q utilizes PERFDB, a statistics catalog derived from benchmarks and execution traces. The framework accommodates sequential operators that relay intermediate answers as context, along with parallel and blend operators that execute models simultaneously and combine their outputs. This initiative tackles the complexities of budgeted LLM deployment for question answering amid nondeterminism and varying resource profiles.