LEAP Protocol Prevents Temporal Leakage in LMS Early-Warning Models
A new study from arXiv introduces LEAP (Leakage-Excluded Early-Availability Protocol), a method to prevent temporal leakage in early-warning models built from Learning Management System (LMS) logs. Temporal leakage inflates reported early performance by using future information not available at prediction time. LEAP enforces cutoff-first truncation before joins and aggregation, and audits feature provenance to exclude post-cutoff evidence. The protocol is instantiated on the Open University Learning Analytics Dataset (OULAD) across weekly cutoffs. Standard learning methods are evaluated using ROC-AUC, PR-AUC, Brier score, and F1@0.5. The work formalizes cutoff-based early outcome prediction under a temporal availability constraint.
Key facts
- LEAP stands for Leakage-Excluded Early-Availability Protocol
- Temporal leakage occurs when prediction uses information unavailable at prediction time
- LEAP enforces cutoff-first truncation prior to joins and aggregation
- The protocol audits feature provenance to prevent post-cutoff evidence
- Evaluated on the Open University Learning Analytics Dataset (OULAD)
- Evaluation metrics include ROC-AUC, PR-AUC, Brier score, and F1@0.5
- The study formalizes cutoff-based early outcome prediction
- Published on arXiv with identifier 2605.25794v1
Entities
Institutions
- arXiv
- Open University