LLM Theorem Provers Fail to Respect Structural Symmetries
A recent preprint on arXiv (2605.22257) demonstrates that formal theorem provers utilizing large language models (LLMs) are extremely responsive to minor changes in how problems are presented, leading to significant discrepancies in proof success rates for semantically identical statements. This highlights a lack of adherence to the structural symmetries present in formal mathematics. The authors propose a framework called "rewriting categories," which is rooted in category theory and encompasses the non-invertible transformations brought about by proof tactics. Within this framework, they define two concepts: proof equivariance, which describes how proof distributions change with rewrites, and success invariance, which mandates that equivalent statements should have the same solution probability. They note that state-based next-tactic provers inherently fulfill proof equivariance by functioning on proof states.
Key facts
- arXiv:2605.22257v1
- LLM-based formal theorem provers are sensitive to superficial variations in problem representation
- Semantically equivalent statements can have drastically different proof success rates
- The paper introduces rewriting categories, a category-theoretic framework
- Rewriting categories capture compositional, generally non-invertible transformations induced by proof tactics
- Two symmetry notions are formalized: proof equivariance and success invariance
- Proof equivariance governs how proof distributions transform under rewrites
- Success invariance requires equivalent statements to be solved with the same probability
- State-based next-tactic provers naturally satisfy proof equivariance
Entities
Institutions
- arXiv