Evergreen: LLM Claim Verification for Semantic Aggregates
A new system called Evergreen addresses the challenge of verifying claims in semantic aggregates produced by LLMs. Semantic aggregation reduces a relation into a natural language aggregate, but these aggregates may contain ungrounded claims. Evergreen recasts claim verification as a semantic query processing task, compiling each claim into a declarative verification query executed on the same engine that generated the aggregate. It employs verification-aware optimizations such as early stopping and relevance sorting to reduce cost and latency by avoiding unnecessary LLM calls. The system handles claims involving quantifiers, groupings, and comparisons over large relations that exceed LLM context windows.
Key facts
- Evergreen is a system for verifying claims in semantic aggregates
- Semantic aggregates are natural language summaries produced by LLMs from relations
- Claims may be ungrounded in the underlying relation
- Verification involves quantifiers, groupings, and comparisons over large relations
- Evergreen compiles claims into declarative semantic verification queries
- Optimizations include early stopping and relevance sorting
- Aims to reduce cost and latency of LLM calls
- Presented in arXiv paper 2604.26180
Entities
Institutions
- arXiv