LLM-Driven Multi-Agent System for Autonomous UI Test Repair
An industrial case study assessed a multi-agent autonomous testing framework that utilizes a large language model, LangGraph orchestration, Playwright for execution, and a RAG knowledge base. This system transitions from human-guided testing to a high-autonomy approach for discovering features and executing tests. Without predefined test objectives, it identified more than 100 testable features across 10 user interface screens and dynamically increased coverage by an extra 15–30 features through runtime DOM analysis. Additionally, it iteratively fixes failing tests autonomously. The analysis covered 300 consecutive reports of autonomous execution, which included 636 distinct test-case executions across 10 distributions.
Key facts
- Multi-agent autonomous testing system evaluated in industrial case study
- Built on LLM with LangGraph orchestration, Playwright execution, and RAG knowledge base
- Discovers over 100 testable features across 10 UI screens without explicit targets
- Dynamically expands coverage by 15–30 features through runtime DOM analysis
- Iteratively repairs failing tests without human intervention
- Analyzed 300 consecutive autonomous execution reports
- 636 individual test-case executions across 10 dis
- Anonymized execution data from production-like enterprise UI testing prototype
Entities
Institutions
- arXiv