PACE-Bench: New Benchmark Tests AI Adaptation to Changing Physics
A new benchmark called PACE-Bench (Physics Adaptation via Code Evolution) has been developed by researchers to assess the recovery capabilities of self-evolving agents when faced with changing execution conditions. This benchmark features 144 adaptation pairs linking source environments to mutated target environments across six physics domains, all maintaining the same goal and interface. Agents must adapt code that works in the source but fails in the target, utilizing feedback from a diagnostic sandbox within a limited number of attempts. The study evaluates ten self-evolving methods from four paradigms, revealing that the benchmark is not fully saturated: Reflexion + Qwen3-14B achieves success on just 35.9% of pairs, whereas GPT-5.5 solves 66.7% of the Statics subset within the full budget. This research fills a gap in current evaluations that usually operate under static conditions without assessing recovery. Further details can be found in the paper titled 'PACE-Bench: Benchmarking Physics Adaptation via Code Evolution in Dynamic Environments,' accessible on arXiv with the identifier 2608.14441.
Key facts
- PACE-Bench is a new benchmark for testing self-evolving agents' adaptation to changing physics.
- It includes 144 source-to-target adaptation pairs across six physics domains.
- Each pair links a source environment to a mutated target environment with the same goal and interface.
- Agents must iteratively adapt a code-driven design using diagnostic sandbox feedback within a limited attempt budget.
- Ten self-evolving methods from four paradigms were compared.
- Reflexion + Qwen3-14B succeeds on only 35.9% of full-benchmark pairs.
- GPT-5.5 solves 66.7% of the Statics subset under the full budget.
- The benchmark remains far from saturated, indicating room for improvement.
Entities
Institutions
- arXiv