CAPRI: A Contract-Aware Proof Repair Workflow for Isabelle Using LLMs
A recent research paper presents CAPRI, a workflow designed for contract-aware proof repairs that utilizes large language models (LLMs) to aid in the discovery of Isabelle proofs. By implementing a machine-readable edit contract through an independent checker, CAPRI fills a significant gap in LLM proof assistance, ensuring that changes comply with established constraints. For auditing, the workflow preserves prompts, proposals, candidate repositories, diagnostics, verdicts, and hashes. The analysis involved five workflows applied to twelve failed proofs from four projects, leading to 180 executions and 138 successful repairs. Among 144 terminal candidates approved by Isabelle, six included modified protected text. The proof-body-only interface achieved 29 valid repairs out of 36 without contract violations, while the full-theory workflow had 31 valid repairs out of 36.
Key facts
- CAPRI is a contract-aware proof repair workflow for Isabelle.
- It uses large language models (LLMs) to help discover Isabelle proofs.
- An independent checker enforces a machine-readable edit contract.
- Prompts, proposals, candidate repositories, diagnostics, verdicts, and hashes are retained for audit.
- Five workflows were evaluated on twelve failed proofs from four developments.
- Three replicates per task and condition gave 180 runs and 138 valid repairs.
- Of 144 terminal candidates accepted by Isabelle, six had modified protected text.
- A proof-body-only interface produced 29/36 valid repairs with no contract violations.
- The full-theory workflow produced 31/36 valid repairs.
- The paper is available on arXiv with identifier 2608.13459.
Entities
Institutions
- arXiv