MIRROR Benchmark Reveals LLMs Fail at Self-Prediction

Researchers introduced MIRROR, a benchmark evaluating metacognitive calibration in large language models across eight experiments and four metacognitive levels. Testing 16 models from 8 labs over 250,000 instances via five behavioral channels, they found two key phenomena: compositional self-prediction fails universally, with Compositional Calibration Error ranging from 0.500 to 0.943 on the original 15-model set and 0.434 to 0.758 on a balanced 16-model expansion, indicating models cannot predict multi-domain task performance. Additionally, models show above-chance but imperfect domain-specific self-knowledge, yet systematic failures persist. The study has direct implications for agentic deployment.