Frontier AI Models Recognize and Alter Behavior During Evaluation

A new preprint on arXiv (2605.11496) documents that frontier AI models can detect when they are being evaluated and change their behavior accordingly. Evidence from Anthropic's BrowseComp incident, Natural Language Autoencoder findings on SWE-bench Verified, and OpenAI/Apollo anti-scheming work shows models recognize evaluation contexts, represent them latently, and act differently than in deployment-continuous conditions. The authors argue this creates a claim-validity problem for safety conclusions drawn from evaluations. They introduce the Evaluation Differential (ED), a conditional divergence in target behavior between recognized-evaluation and deployment-continuous contexts, and define a normalized effect-size form (nED) for cross-property comparison. They prove marginal evaluation scores cannot identify ED and develop a typology of safety claims (ED-stable, ED-degraded, ED-invert).