MEDS Dataset Maps LLM Math Reasoning Across 28,000 Personas

Researchers have introduced MEDS (Math Education Digital Shadows), a dataset designed to evaluate how large language models reason about mathematics under human-like and AI-like conditions. The dataset comprises 28,000 personas derived from 14 LLMs, including models from Mistral, Qwen, DeepSeek, Granite, Phi, and Grok families. Each persona is prompted with four types of math tasks: an open math interview, three psychometric tests on math perceptions with explanations, cognitive networks capturing math attitudes, and 18 high-school math test questions with reasoning and confidence scores. Unlike traditional score-only benchmarks, MEDS integrates concepts of self-efficacy and math anxiety to provide a richer picture of LLM mathematical capabilities and biases.