GTA-2 Benchmark Introduced for Evaluating General Tool Agents in Real-World Workflows

A novel benchmark, named GTA-2, has been introduced to assess General Tool Agents, aiming to bridge the gap between existing tool-use evaluations and the demands of real-world productivity. This benchmark is designed to transition agent development from merely executing instructions to managing intricate workflows, prioritizing authenticity by incorporating genuine user queries, utilized tools, and multimodal contexts. GTA-2 includes two primary elements: GTA-Atomic, which evaluates short-term, closed-ended tool-use accuracy, and GTA-Workflow, which presents long-term, open-ended tasks for authentic end-to-end execution. To appraise these open-ended tasks, researchers have suggested a recursive checkpoint-based evaluation method that breaks down goals into measurable sub-goals for a cohesive assessment. This benchmark seeks to address the shortcomings of current methods that depend on AI-generated queries, artificial tools, and restricted system-level coordination. This research, shared on arXiv under the identifier arXiv:2604.15715v1, marks a significant leap in general-purpose agent evaluation, evolving from basic tool usage to thorough workflow analysis. It builds on previous GTA benchmark studies while offering notable advancements in evaluation techniques for complex, real-world applications.