Systematizing Safety and Security Threats of Computer-Using Agents
A new survey paper on arXiv (2505.10924) systematizes knowledge on the safety and security threats of Computer-Using Agents (CUAs), which are LLM-based systems that autonomously interact with graphical user interfaces. The paper conducts a comprehensive literature review to define CUAs for safety analysis, identify vulnerabilities in LLM-driven reasoning, and address risks from integrating multiple software components and multimodal inputs. The work aims to map the emerging threat landscape as CUAs evolve from basic prototypes to sophisticated agents capable of navigating desktop apps, web pages, and mobile apps.
Key facts
- arXiv paper 2505.10924 presents a survey on safety and security threats of Computer-Using Agents.
- CUAs are LLM-based systems that emulate human-like operations in graphical user interfaces.
- The survey systematizes knowledge across four research objectives, including defining CUAs for safety analysis.
- Vulnerabilities arise from LLM-driven reasoning and integration of multiple software components and multimodal inputs.
- The paper conducts a comprehensive literature review on CUA safety and security.
- CUAs can autonomously perform tasks like navigating desktop applications, web pages, and mobile apps.
- The title references JARVIS and Ultron to contrast helpful vs. harmful AI agents.
- The paper is categorized as a replace-cross announcement type.
Entities
Institutions
- arXiv