DIANOIA: New Diagnostic Framework for Multi-Agent LLM Systems
A new diagnostic framework named DIANOIA has been developed by researchers for multi-agent LLM systems, breaking down reasoning gain into three quantifiable components: coverage, fidelity, and synthesis. This framework features a protocol designed to pinpoint bottleneck channels for various tasks and is realized as a multi-agent system with diverse proposers, verification grounded in execution, and iterative synthesis. In evaluations on benchmarks such as GSM8K, AIME-2025, MBPP, and BFCL-SP, DIANOIA surpasses robust multi-agent baselines while adhering to equivalent token budgets, effectively leading the Pareto front. This research fills a gap in the field by providing a diagnostic framework with measurable primitives and verifiable predictions.
Key facts
- DIANOIA decomposes multi-agent reasoning gain into coverage, fidelity, and synthesis.
- Each channel is empirically measurable.
- A diagnostic protocol identifies bottleneck channels for any given task.
- The system includes role-diverse proposers, execution-grounded verification, and iterative synthesis.
- Tested on GSM8K, AIME-2025, MBPP, and BFCL-SP benchmarks.
- Outperforms strong multi-agent baselines under matched token budgets.
- Dominates the Pareto front.
- Addresses the gap in diagnostic frameworks for multi-agent LLM systems.
Entities
Institutions
- arXiv