VeriTrip: New Benchmark Tests Travel Planning Agents on Unstructured Web Data
A new benchmark called VeriTrip has been developed by researchers to assess travel planning agents, moving past traditional API-focused methods to evaluate their resilience on unstructured multimodal web data. This benchmark tackles significant cognitive challenges, including information noise, conflicting facts from various sources, and the integration of visual perception into logical planning. VeriTrip features a Multimodal Retrieval Base (MRB) sourced from real-world information, compelling agents to independently manage queries across diverse datasets. Additionally, it incorporates a synchronized Verifiable Knowledge Base to support evidence-based reasoning. This research is detailed in a paper available on arXiv under ID 2605.28683.
Key facts
- VeriTrip is a verifiable benchmark for travel planning agents.
- It shifts evaluation from API-centric to evidence-grounded reasoning over unstructured web corpora.
- The benchmark includes a Multimodal Retrieval Base (MRB) from real-world sources.
- It addresses information noise, factual contradictions, and visual perception grounding.
- A synchronized Verifiable Knowledge Base supports the evaluation.
- The paper is available on arXiv with ID 2605.28683.
Entities
Institutions
- arXiv