Language Models Detect and Identify Activation Perturbations with High Accuracy

Research demonstrates that large language models can identify when their internal activations have been altered. Experiments involved applying either dropout-like masking or Gaussian noise to specific sentences within the models' processing. Models from the Llama, Olmo, and Qwen families, ranging from 8B to 32B parameters, were tested. These systems could not only detect the presence of a perturbation but also pinpoint its location within a sequence. When presented with a multiple-choice format, models achieved high, sometimes perfect, accuracy in answering which sentence was modified. Furthermore, the models learned to distinguish between the two types of interference—dropout and noise—when provided with contextual examples. A notable finding showed that Qwen's ability to identify the perturbation type in a zero-shot setting improved as the strength of the interference increased. This capability, however, diminished if the instructional labels within the context were swapped or made incorrect. The study, detailed in the arXiv preprint 2604.17465v1, provides evidence that language models possess a form of introspective awareness regarding their own computational states.