Govllm: Continuous LLM Compliance Monitoring via Governance from Metrics
A new framework, govllm, proposes continuous compliance monitoring for large language models (LLMs) using runtime observability rather than static audits. The approach, called governance from metrics, derives regulatory compliance as a continuous signal from production systems. It addresses the EU AI Act's requirement for ongoing human oversight and detection of emergent behavioral drift. Govllm implements a governance-driven routing architecture where model selection depends on accumulated compliance scores, not just latency or cost. A panel of regulatory judges—LLM evaluators specialized per criterion (EU AI Act, GDPR, ANSSI, accessibility)—assesses compliance. The system monitors inter-judge disagreement to ensure robust governance.
Key facts
- Govllm is an open-source framework for continuous LLM compliance monitoring.
- Governance from metrics derives compliance as a continuous signal from runtime observability.
- The framework addresses the EU AI Act's demand for ongoing human oversight.
- Model selection is based on accumulated compliance scores, not latency or cost.
- A panel of regulatory judges evaluates compliance per criteria including EU AI Act, GDPR, ANSSI, and accessibility.
- Inter-judge disagreement is monitored to ensure robust governance.
- The approach treats compliance as a continuous property, not a binary audit-time verdict.
- The framework detects emergent behavioral drift in deployed systems.
Entities
Institutions
- EU AI Act
- GDPR
- ANSSI