IntelliAudit: Multi-Agent LLM System for IT Audit Evidence Evaluation
A recent paper published on arXiv (2608.07688) presents IntelliAudit, a multi-agent system grounded in retrieval that assesses IT audit evidence through large language models. This innovative system tackles the complexities of automating IT audits, which involve determining if diverse organizational evidence meets semantic security and compliance standards. IntelliAudit extracts pertinent artifacts from an evidence corpus, formulates assessments based on evidence, contests negative findings, resolves disputes, and offers auditor-oriented recommendations complete with cited evidence, rationale, analysis of missing evidence, and remediation advice. Built on ISO/IEC 27001, the system was tested across various simulated organizations, utilizing expert auditor evaluations and user feedback on audit readiness. The paper emphasizes the challenges of automating audit conclusions due to the scattered nature of evidence across various documents and the necessity for evidentiary sufficiency over simple keyword matching. IntelliAudit is designed to support auditors by delivering clear, evidence-driven recommendations.
Key facts
- IntelliAudit is a retrieval-grounded multi-agent system for IT audit evidence evaluation.
- It uses large language models to judge semantic security and compliance controls.
- The system retrieves relevant artifacts, generates assessments, challenges adverse findings, and adjudicates disagreements.
- It produces auditor-facing recommendations with cited evidence, rationale, missing-evidence analysis, and remediation guidance.
- IntelliAudit is instantiated on ISO/IEC 27001.
- Evaluation was conducted across multiple simulated organizations using expert auditor review and audit-readiness user feedback.
- The paper is available on arXiv with ID 2608.07688.
- The system addresses the challenge of evidence distributed across policies, records, spreadsheets, and operational artifacts.
Entities
Institutions
- arXiv