EarlyDx: New Benchmark for Open-Ended ED Diagnosis from Admission Data
A team of researchers has introduced EarlyDx, a new benchmark designed to improve early diagnosis in emergency departments, based on data from 154,834 cases in MIMIC-IV. Unlike typical benchmarks that restrict predictions to specific codes and ignore free-text notes, EarlyDx uses only information available at the time of admission, relying on diagnoses made during the emergency visit. An LLM auditor evaluates each free-text label as fully supported, partially supported, or unsupported, focusing mainly on fully supported labels for assessment. The study revealed that no evaluated system—be it general, medical-specific, or post-trained—could consistently generate a fully supported diagnosis for every case. You can find the research on arXiv with the identifier 2607.28788.
Key facts
- EarlyDx is a benchmark for open-ended early diagnosis in emergency departments.
- It is built from 154,834 emergency department encounters in MIMIC-IV.
- Each encounter is restricted to records available at admission time.
- Supervision uses diagnoses recorded during the ED encounter, not discharge diagnoses.
- An LLM auditor verifies free-text labels as supported, partially supported, or unsupported.
- Primary evaluation scores only fully supported labels.
- No evaluated system synthesized a fully supported diagnosis for all cases.
- The study is available on arXiv under identifier 2607.28788.
Entities
Institutions
- arXiv