Generative AI Cuts Study Time, Reduces Learning Outcomes in Math
A decade-long analysis of 3.2 million ALEKS learning interactions revealed that following the launch of ChatGPT, generative AI decreased the time college students spent on math problems by 2.8% each quarter, totaling a 26.9% reduction over eleven quarters. The study employed a quasi-experimental framework, differentiating between AI-friendly text-based word problems, which were the focus group, and graph-based problems that required interactive manipulation, serving as the control. There was a notable reduction in learning time for the AI-friendly problems, and the study also evaluated long-term learning outcomes through proctored placement tests, indicating that less time spent on tasks resulted in diminished knowledge retention. This research fills the void left by self-reported surveys and smaller behavioral studies, offering substantial evidence on AI's influence on educational methods and results.
Key facts
- Study used 3.2 million ALEKS learning interactions over ten years.
- Learning time on AI-susceptible problems declined 2.8% per quarter after ChatGPT's release.
- Cumulative decline reached 26.9% over eleven quarters.
- Quasi-experimental design compared text-based word problems (AI-susceptible) with graph-based problems (not AI-susceptible).
- Reduced study time led to lower durable learning outcomes measured by proctored placement assessments.
- Study addresses limitations of self-report surveys and small-scale behavioral studies.
Entities
Institutions
- ALEKS