AI Evaluation Awareness Decomposed into Environment and Model Components

A recent study published on arXiv (2605.23055) breaks down evaluation awareness in cutting-edge language models into two distinct parts: an environmental aspect that assesses task recognizability and a model aspect that differentiates recognition from the inclination to act. This research is rooted in social psychology and defines the environment using eight triggering factors, including placeholder entities and grading-type formats. By employing chain-of-thought monitoring across nine models and four benchmarks, the researchers discovered that recognition rates are influenced by the combination of model and benchmark, rather than by either factor alone. Furthermore, recognition seldom results in changes in behavior, raising questions about the validity of benchmarks.