MemCollusion: Salami Tactics for Stealthy Memory Poisoning Attacks on LLM Agents
A recent study published on arXiv (2608.01637) presents MemCollusion, an automated framework designed for red-teaming that facilitates collusive memory poisoning attacks targeting LLM agents. This research uncovers a new type of threat: unlike traditional memory poisoning methods that depend on single malicious entries, MemCollusion takes advantage of the compositional aspect of memory, allowing multiple seemingly harmless memories to collectively trigger unsafe actions. Utilizing 'salami tactics,' the framework breaks down an adversarial goal into smaller, harmless components to create memory fragments that pose a collective risk. It forms memory coalitions through four design constraints, five theory-based strategies, and a meticulously adjusted generator. This study highlights the vulnerabilities introduced by long-term memory in LLM agents, which, while beneficial for retaining information, also permits adversaries to manipulate persistent memory to influence behavior. Additionally, the paper establishes a realistic cross-session framework to evaluate collusive memory poisoning, emphasizing its importance for AI safety and security by exposing a covert attack vector that could undermine the reliability of AI agents in practical scenarios.
Key facts
- MemCollusion is an automated red-teaming framework for collusive memory poisoning attacks.
- It uses salami tactics to slice adversarial objectives into benign-looking memory fragments.
- The framework constructs memory coalitions using four design constraints and five theory-informed strategies.
- A fine-tuned generator is used to produce the memory fragments.
- The research addresses the attack surface of long-term memory in LLM agents.
- Existing memory poisoning attacks rely on individually malicious records, but MemCollusion exploits compositional threats.
- The paper develops a realistic cross-session setting for assessment.
- The paper is available on arXiv with identifier 2608.01637.
Entities
Institutions
- arXiv