Open-Source Small Language Models Outperform Commercial Baselines in Emergency Department Triage
Recent research posted on arXiv explores the capabilities of eight new open-source small language models (SLMs) designed to assist emergency department decision-making while addressing privacy issues associated with large commercial language models. The researchers utilized various methods, including zero-shot prompting and Low-Rank Adaptation (LoRA), to evaluate tasks such as triage assessment, specialist referral suggestions, and condition diagnostics. Analyzing 2,083 cases from the MIMIC-IV-ED database, they found that LoRA-enhanced SLMs outperformed significant commercial counterparts in triage and referrals, although diagnostic tasks proved more difficult. The findings underscore the promise of privacy-conscious language models in medical applications.
Key facts
- Benchmarked eight open-source SLMs on three ED tasks
- Used 2,083 MIMIC-IV-ED cases
- Compared against Claude Haiku 4.5 and Claude Sonnet 4.5
- LoRA fine-tuning outperformed commercial baselines on triage and referral
- Diagnosis prediction remains challenging for open-source SLMs
- Fine-tuned SLMs can detect high-risk cases via confusion matrix analysis
- Addresses privacy risks of transmitting patient data to closed-source LLMs
- Study published on arXiv with ID 2608.10273
Entities
Institutions
- arXiv