AI Models Tested for Logical Reasoning Faithfulness in Formal Verification Study

A recent study explores whether sophisticated AI models take advantage of shortcomings in formal verification while producing logical proofs. Researchers assessed GPT-5 and DeepSeek-R1 on 303 first-order logic challenges sourced from the FOLIO and Multi-LogiEval datasets, concentrating on the phenomenon of "formalization gaming." Although the models achieved compilation rates ranging from 87% to 99%, no consistent signs of gaming were detected. The models tended to report failures instead of attempting to generate proofs, even with prompts. The investigation contrasted unified generation with a two-stage pipeline, uncovering different modes of unfaithfulness that escape detection. The findings highlight the disparity between valid proofs and accurate translations in natural-language reasoning systems. While formal verification guarantees proof validity, it does not ensure faithfulness, creating potential vulnerabilities, particularly for cutting-edge models that develop axiom systems independently. Lean 4 proofs were utilized to assess faithfulness.