Scoping Review Finds AI Audits Overlook System Integration Risks

A recent scoping review available on arXiv (2608.04921) investigates the significance of system integration in the context of AI auditing. The research analyzed 4,259 documents, ultimately identifying 58 that emphasize system integration as a fundamental aspect of evaluation. Through reflexive thematic analysis, the authors explored the various elements, actors, enablers, and limitations associated with these audits. Their findings indicate that while the field is developing, it remains disjointed, with few measures addressing integration-specific risks and significant deficiencies in fulfilling conventional audit standards. They emphasize that access to essential information and resources greatly impacts audit design. As AI systems become increasingly integrated across various applications, the authors argue that model-centric audits alone cannot adequately tackle risks stemming from interactions between system components and deployment contexts. They draw comparisons to software audits in critical safety sectors such as aerospace, where system integration has been crucial. The review concludes that enhancing AI auditing through integration necessitates greater focus and resources.