ATBench Extends Trajectory Safety Evaluation to OpenClaw and Codex
ATBench, a benchmark for trajectory-level safety evaluation and diagnosis in agent systems, has been extended with two domain-customized versions: ATBench-Claw and ATBench-Codex. These extensions adapt the benchmark to the OpenClaw and OpenAI Codex/Codex-runtime settings, respectively. The adaptation mechanism involves analyzing each new execution environment, customizing a three-dimensional Safety Taxonomy covering risk source, failure mode, and real-world harm, and using that taxonomy to define the benchmark specification for the shared ATBench construction pipeline. This extensibility is crucial as agent frameworks remain architecturally stable while their concrete settings, tool ecosystems, and product capabilities evolve rapidly. The work is detailed in arXiv paper 2604.14858.
Key facts
- ATBench is a benchmark for trajectory-level safety evaluation and diagnosis in agent systems.
- ATBench-Claw extends ATBench to the OpenClaw setting.
- ATBench-Codex extends ATBench to the OpenAI Codex/Codex-runtime setting.
- The adaptation uses a three-dimensional Safety Taxonomy: risk source, failure mode, real-world harm.
- The benchmark specification is consumed by a shared ATBench construction pipeline.
- Agent frameworks remain stable architecturally while execution settings evolve.
- The paper is available on arXiv with ID 2604.14858.
- The extensions are domain-customized for diverse execution settings.
Entities
Institutions
- OpenAI
- arXiv