New Taxonomy Localizes Agent Failures to Interactions
A recent publication on arXiv (2607.28802) presents a taxonomy focused on interactions to identify failures in AI agent systems. The authors contend that current evaluations often simplify agent failures to system-wide results, masking the actual causes of issues and the most suitable interventions. This leads to a 'repair-assignment problem,' where a single observable failure might necessitate model adjustments, harness modifications, environmental changes, or benchmark corrections, depending on its origin. The taxonomy categorizes 41 failure modes by linking each to an edge connecting two components (models, harnesses, users, tools, memory, environments) and a fault side indicating the repair's location. This framework aims to establish a common basis for failure analysis, moving past benchmark-specific classifications. The paper can be accessed at https://arxiv.org/abs/2607.28802.
Key facts
- Paper arXiv:2607.28802 introduces an interaction-centric taxonomy for agent failures.
- The taxonomy addresses the repair-assignment problem in AI agent systems.
- It organizes 41 failure modes.
- Each failure mode is assigned to an edge between two components and a fault side.
- Components include models, harnesses, users, tools, memory, and environments.
- The taxonomy aims to localize failures to specific interactions.
- It provides a shared structure beyond benchmark-specific taxonomies.
- The paper is available on arXiv.
Entities
Institutions
- arXiv