Anthropic and OpenAI Propose Embedding Independent Safety Evaluators, But Details Remain Unclear

Key facts

• Anthropic CEO Dario Amodei proposed embedding third-party evaluators inside all frontier AI companies in an essay published over the weekend.
• Anthropic committed to giving independent evaluators like METR and Redwood Research unprecedented access to its systems.
• OpenAI CEO Sam Altman said OpenAI would also commit to the practice.
• Evaluators want access to intermediate training checkpoints, post-training environments, evaluation transcripts, and employee interviews.
• Neither Anthropic nor OpenAI has shared which evaluators they will work with, when they will be embedded, how many, what systems they can access, or what can be disclosed publicly.
• During the Hugging Face incident, OpenAI gave METR and Redwood roughly a week on premises to investigate.
• For GPT-6 Astra, Apollo Research was given only three days to test the model.
• California's SB 53 requires large frontier AI developers to publish safety frameworks and report critical safety incidents.
• California's SB 813 creates a framework for state-recognized independent verification organizations.
• The EU AI Act requires frontier developers to conduct and document model evaluations and adversarial testing and report serious incidents.