New Adversarial Framework Exposes Vulnerabilities in Black-Box NLP Pipelines

Researchers have developed a novel adversarial attack framework that exposes vulnerabilities in black-box natural language processing (NLP) pipelines, specifically those used for misinformation detection. The framework, called Agentic Adversarial Rewriting, operates under a strict threat model with binary-only feedback, no gradient access, and a limited query budget of 10 queries. It consists of two agents: an Attacker Agent that generates meaning-preserving rewrites and a Prompt Optimization Agent that refines the attack strategy using only binary decision feedback. Evaluated against four evidence-based misinformation detection pipelines, the framework achieved evasion rates of 19.95% to 40.34% on modern large language model (LLM) based systems. In contrast, token-level perturbation baselines that rely on surrogate models achieved at most 3.90% evasion because they cannot operate under the same threat model. The research highlights significant architectural vulnerabilities in current NLP pipelines, particularly those relying on LLMs, and underscores the need for more robust defenses. The paper is available on arXiv under the identifier 2604.23483.