Veroic: Adaptive Inference Control for Black-Box LLMs via Verifiable Observations
A new framework called Veroic (Verifiable Observations for Risk-aware Inference Control) addresses the budgeted sequential decision problem in black-box large language model services. The system decides per request whether a default low-cost response is reliable or if additional computation is needed to improve quality. It formulates request-time control as a partially observable Markov decision process (POMDP) to handle partial observability and sequential budget coupling. A lightweight verifiable observation channel is constructed from input-output pairs by aggregating heterogeneous quality signals into a belief state.
Key facts
- Proposes Veroic framework for adaptive inference control in black-box LLM settings
- Formulates request-time control as a partially observable Markov decision process
- Captures partial observability and sequential budget coupling
- Constructs a lightweight verifiable observation channel from input-output pairs
- Aggregates heterogeneous quality signals into a belief state
- Addresses the budgeted sequential decision problem for each request
- Decides whether default low-cost response is reliable or additional computation needed
- Published on arXiv with ID 2604.27536
Entities
Institutions
- arXiv