Agent Decomposition Degrades Policy Compliance in Financial AI
Fiducia-bench is a new benchmark designed to evaluate how well financial AI agents can be governed. It looks at whether these agents know when to escalate or hold back and how well they keep track of their actions. A recent study, shared on arXiv (2608.16055), investigates an important question: does breaking an agent into components hurt its governance? The findings indicate that it does, particularly because information crucial for decision-making often gets lost when it's passed between parts. In tests with 626 episodes across 100 KYC/AML tasks, a 32B model showed no loss of information in a single-loop setup, 56% loss in a fixed pipeline, and 85% in an orchestrator-subagent model. A stronger model, gpt-4.1-mini, had a 3-6% loss in similar tests. These results highlight the need for balance between modular design and compliance in AI systems for regulated industries.
Key facts
- Fiducia-bench is a new benchmark for the governability of financial agents.
- The benchmark assesses escalation, abstention, and audit trail compliance.
- The study asks whether decomposing an agent into components degrades governance.
- Policy-relevant facts are attenuated at handoff boundaries between components.
- Experiment involved 626 episodes across 100 KYC/AML task variants.
- Two models were tested: a 32B open-weights model and gpt-4.1-mini.
- Three architectures were compared: single-loop, fixed pipeline, orchestrator-subagent.
- The 32B model attenuated 0% of facts in single-loop, 56% in fixed pipeline, 85% in orchestrator-subagent.
- gpt-4.1-mini attenuated 3-6% of facts under the same conditions.
- The paper is available on arXiv with ID 2608.16055.
Entities
Institutions
- arXiv