CITE Algorithm for Anytime-Valid Inference in LLM Self-Consistency
A novel statistical approach known as Certification by Intersection-union Testing with E-processes (CITE) facilitates the anytime-valid certification of a target answer as the distinct mode of a large language model's response distribution. This algorithm effectively manages false certification at any specified level, regardless of arbitrary data-driven stopping, and does not necessitate prior knowledge of the answer category set. Additionally, it offers a stopping-time rate that is independent of the category set size and aligns with matching minimax lower bounds, subject to constants in the primary regime. The findings are detailed in arXiv:2605.05873.
Key facts
- CITE stands for Certification by Intersection-union Testing with E-processes
- It provides anytime-valid statistical inference for LLM self-consistency
- Controls false certification at any prescribed level
- Does not require prior knowledge of answer category set
- Proves category-set-size-free stopping-time rate
- Establishes matching minimax lower bounds up to constants
- Target is to certify a prespecified answer as unique mode of response distribution
- Published on arXiv with ID 2605.05873
Entities
Institutions
- arXiv