LLM Jailbreak Vulnerability Linked to Internal Layer Features

A recent study published on arXiv indicates that the success of jailbreak attempts in large language models (LLMs) is influenced by specific internal characteristics rather than merely the prompts used. The researchers introduce a three-step process for Gemma-2-2B utilizing the BeaverTails dataset. They extract tokens aligned with concepts from adversarial outputs, implement feature-grouping techniques (including cluster, hierarchical-linkage, and single-token-driven methods) to pinpoint SAE feature subgroups throughout all 26 layers, and enhance the model's performance by boosting the most significant features. Findings reveal that layers 16-25 exhibit increased susceptibility.