New Framework for Anytime-Valid Risk Control in LLM Deployments
A research paper introduces Conformal Selective Acting (CSA), a framework for controlling selective risk in real-time LLM deployments. The method addresses the challenge of providing safety certificates for RLVR-trained models operating within strict per-deployment error budgets, without pooling data across deployments. Existing methods like offline conformal risk control require exchangeability, online methods bound only long-run averages, and A-RCPS controls marginal rather than selective risk. CSA fills this gap by using an e-process per threshold, ensuring anytime-pathwise validity for selective risk. The framework is designed for regulated organizations needing per-round safety guarantees.
Key facts
- Paper introduces Conformal Selective Acting (CSA) for anytime-valid risk control.
- CSA targets RLVR-trained LLMs deployed in regulated settings.
- Existing methods fail to provide per-deployment, per-round safety certificates.
- CSA uses e-process per threshold for selective risk control.
- Framework ensures anytime-pathwise validity.
- Addresses limitations of offline and online conformal methods.
- Designed for adaptive, online-updated streams.
- No pooling across deployments is required.
Entities
Institutions
- arXiv