Certified Runtime Safety for Tool-Using Agents
A recent publication on arXiv (2607.22868) introduces a theory concerning certified runtime safety for agents that utilize tools. The researchers distinguish three fundamental inquiries: the safety policies enforceable by a deterministic gate with predetermined oracle predicates, the optimal balance between false blocks and misses under a constant exogenous law, and the determination of the closed-loop frontier when blocking influences future proposals. They demonstrate that policy nontriviality is undecidable with two decrementable counters, yet remains in PSPACE for a separable monotone fragment. Additionally, they reveal that Neyman-Pearson establishes the precise false-block/miss frontier, while conformal calibration offers finite-sample marginal certificates. Furthermore, bounded representation attacks enhance robustness.
Key facts
- arXiv paper 2607.22868
- Theory of certified runtime safety for tool-using agents
- Separates three questions: enforceability, optimality, closed-loop frontier
- Deterministic gate enforces nonempty safety policies whose good prefixes its register model recognizes
- Policy nontriviality undecidable with two decrementable counters
- Policy nontriviality in PSPACE for separable monotone fragment
- Neyman-Pearson gives exact false-block/miss frontier
- Conformal calibration gives finite-sample marginal certificate
- Bounded representation attacks add robustness margin
Entities
Institutions
- arXiv