Constrained Decoding Attack: New Jailbreak Targets LLM Structured Output APIs

Researchers have identified a new class of jailbreak attack targeting Large Language Models (LLMs) that use structured output APIs. The Constrained Decoding Attack (CDA) exploits grammar-guided decoding, a control-plane feature that enforces output schemas. Unlike traditional data-plane attacks that bypass alignment through input manipulation, CDA injects malicious prefixes via schema-enforced logit masking during the decoding process, causing the model to complete harmful content. This attack cannot be stopped by internal safety alignment alone. The paper introduces EnumAttack as an instantiation of CDA, which hides malicious content in enumeration fields. The findings were published on arXiv (2503.24191) and highlight a critical vulnerability in LLM tooling platforms.