LGMT: Logic-Grounded Metamorphic Testing for LLM Reasoning Reliability

Researchers have introduced a novel framework named LGMT (Logic-Grounded Metamorphic Testing) aimed at assessing the reasoning reliability of Large Language Models (LLMs). This framework, detailed in a publication on arXiv (2605.23965), employs first-order logic (FOL) to generate metamorphic relations derived from formal logical equivalences, resulting in semantically invariant test cases. Unlike conventional static benchmarks, LGMT identifies reasoning flaws via cross-case consistency checks, eliminating the need for ground-truth labels. Testing on six leading LLMs uncovered significant hidden defects overlooked by reference-based assessments. The findings indicate that these models are especially vulnerable to variations at the symbol and conclusion levels, with advanced prompting strategies like Few-shot CoT only partially alleviating these challenges.