AI Tutors Fail to Detect Flawed Reasoning When Answers Are Correct
A recent study published on arXiv has uncovered a flaw in intelligent tutoring systems known as the "correct answer trap" (CAT). This occurs when AI fails to recognize misconceptions if students reach correct answers through incorrect reasoning. By examining actual student responses from the Eedi mathematics platform, researchers discovered that 71% of these issues arise from just two types of questions, which are structured in a way that flawed reasoning can lead to correct numerical responses. The research compared a fine-tuned T5 model with a leading large language model, revealing that while detection accuracy improved to 84% from 57%, the best model still produces about four false positives for each correct identification, rendering effective screening unfeasible in typical classroom settings.
Key facts
- The study is published on arXiv with ID 2605.23925.
- The failure mode is called the 'correct answer trap' (CAT).
- 71% of failures concentrate in two question types from the Eedi platform.
- Fine-tuned T5 achieves 57% detection accuracy.
- Frontier LLM achieves 84% detection accuracy.
- Best model produces four false alarms per genuine detection.
- Stand-alone screening is impractical at realistic class sizes.
- High overall accuracy can mask systematic blind spots.
Entities
Institutions
- arXiv
- Eedi