LLM Agents Overcommit on Blocked Tasks in New Audit Framework
Researchers have unveiled the Support-State Triage Audit (SSTA-32), designed to assess how well large language model (LLM) agents can recognize the sources of task interruptions. The tool categorizes requests into four types: COMPLETE, CLARIFY, REQUEST SUPPORT, and ABSTAIN. The study analyzed a prominent model using various prompting techniques, including Direct and Action-Only methods, and implemented a dual-persona auto-auditing system. Findings indicate a significant tendency for overcommitment on tasks, with a striking 41.7% rate of executing incomplete requests. This work is available on arXiv, under the identifier 2604.16752.
Key facts
- SSTA-32 is a matched-item diagnostic framework
- Four support states: Complete, Clarifiable, Support-Blocked, Unsupported-Now
- Four prompting conditions: Direct, Action-Only, Confidence-Only, Preflight Support Check
- Evaluation uses Dual-Persona Auto-Auditing with deterministic heuristic scoring
- Default execution overcommitment rate is 41.7% on non-complete tasks
- Paper available on arXiv: 2604.16752
- Study addresses whether agents can diagnose task blockage before acting
- Current agent evaluations largely reward execution on fully specified tasks
Entities
Institutions
- arXiv