Agent Security Must Be a Systems Problem, Not Just Model Robustness

A new paper on arXiv argues that we should think about AI security as a broader systems problem. The authors, who are experts in cybersecurity and have backgrounds in operating systems and adversarial machine learning, point out that current efforts to strengthen AI models aren’t enough. They propose key principles from their extensive research to develop more secure AI systems. The paper examines eleven real-world attacks on AI agents, showing that if we had used these systems principles, those attacks could likely have been prevented. It also emphasizes the research hurdles that still need to be tackled.