AI Deployment Needs Calibrated Verification, Not Mechanistic Interpretability

A recent study published on arXiv suggests that the implementation of AI in critical areas such as healthcare, credit, employment, and criminal justice should not rely solely on mechanistic interpretability. The authors advocate for a calibrated verification framework, where authorization is specific to each domain, independently verifiable, subject to post-release monitoring, and can be held accountable, contested, and revoked. They highlight the variability in model performance across different tasks, emphasizing that authorization should be tied to particular applications rather than the model itself. Furthermore, the authors point out that societies have historically managed opaque expertise through means such as credentials, oversight, liability, appeals, and revocation, rather than through detailed mechanistic explanations. The paper notes a significant 53-percentage-point disparity between mechanistic understanding and deployment authority.