MeDial-Speech Dataset for Medical AI Consultations
A new dataset named MeDial-Speech has been introduced to aid in the training and assessment of medical AI systems (Med-AIs) during patient consultations. This dataset, which encompasses over 111 hours of unaugmented speech data, is derived from interactions between robots and patients, as well as between doctors and patients, in realistic settings. It addresses four specific health issues: Lewy body dementia, heart failure, shoulder pain, and angina. Additionally, it features a dialogue benchmark with 20 sentence selection choices to evaluate three advanced LLMs: GPT-5 mini, DeepSeek-V3, and Claude Sonnet 4. Findings indicate that Claude Sonnet 4 achieves the highest accuracy rate of 71.1% when utilizing manual transcriptions.
Key facts
- MeDial-Speech is a novel speech dataset for medical consultations.
- Dataset collected from robot-patient and doctor-patient dialogues.
- Contains 111+ hours of speech data without augmentation.
- Covers four health conditions: Lewy body dementia, heart failure, shoulder pain, angina.
- Includes a dialogue benchmark with 20 sentence selection options.
- Evaluates GPT-5 mini, DeepSeek-V3, and Claude Sonnet 4.
- Claude Sonnet 4 achieves 71.1% accuracy on sentence selection.
- Paper published on arXiv with ID 2605.26747.
Entities
Institutions
- arXiv