ReLoop: Structured Modeling and Behavioral Verification for Reliable LLM-Based Optimization
ReLoop, a newly introduced framework, tackles the discrepancy between feasibility and correctness in optimization code generated by LLMs, where silent failures may yield solver-feasible yet semantically flawed formulations. This discrepancy can be as high as 90 percentage points in compositional issues. ReLoop incorporates two interrelated strategies: structured generation, which breaks down code creation into a four-step reasoning process (understand, formalize, synthesize, verify), and behavioral verification, which identifies mistakes by evaluating how formulations respond to changes in solver parameters. These strategies complement each other in terms of error structure, with structured generation contributing the most significant improvements in compositional challenges.
Key facts
- LLMs can translate natural language into optimization code but may produce silent failures.
- Silent failures create a feasibility-correctness gap reaching 90 percentage points on compositional problems.
- ReLoop introduces structured generation with a four-stage reasoning chain.
- ReLoop introduces behavioral verification using solver-based parameter perturbation.
- Behavioral verification requires no ground truth.
- Structured generation and behavioral verification are complementary by error structure.
- Structured generation drives the largest gains on compositional problems.
- The paper is available on arXiv under identifier 2602.15983.
Entities
Institutions
- arXiv