AdmitOR: A Label-Free Admission Gate for LLM-Based Optimization Modeling
A new study on arXiv (2608.15565v1) introduces AdmitOR, a mechanism aimed at helping experience-learning agents in optimization tasks. It addresses the challenge of label-free certification, which means agents need to prove their abilities without having the correct answers upfront. Current methods show inconsistency: in a label-blind scenario of 300 problems, allowing all viable models results in about 25% being flawed, while agreeing on a single instance can hide divergences at others. AdmitOR uses calibrated external behavioral data to evaluate candidates from three model types and strategies. A cross-family clique represents the agreement among value-function traces, with a set threshold guiding whether to accept, abstain, or escalate. However, while the false-discovery criterion works well with calibration data, it struggles in real-world conditions.
Key facts
- Paper arXiv:2608.15565v1 introduces AdmitOR, an admission gate for LLM-based optimization modeling.
- AdmitOR addresses label-free certification, where agents must verify skills without known answers.
- Existing label-free alternatives are unreliable: on a 300-problem label-blind stream, admitting every executable model poisons roughly one admission in four.
- Single-instance agreement accepts models that match at one value but differ elsewhere.
- AdmitOR uses calibrated external behavioral evidence from candidates across three model families, prompting strategies, and solver stacks.
- Instances are resampled from an extracted parameter domain.
- Agreement across value-function traces is summarized by a cross-family clique.
- A calibrated threshold returns accept, abstain, or escalate.
- The preregistered false-discovery criterion holds on calibration data but not on the wild stream.
Entities
Institutions
- arXiv