LLMs Match Supervised Models in Clinical Action Extraction

A recent study published on arXiv investigates the efficacy of zero-shot and few-shot large language models (LLMs) in extracting clinical actions from discharge summaries, utilizing the CLIP dataset. The authors propose a two-stage extraction method that breaks down narrative discharge notes into specific actionable tasks through staged prompting. They conduct a thorough evaluation of generative LLMs, contrasting their performance with that of supervised BERT-based models tailored for specific tasks, while also examining inconsistencies in annotations. Findings indicate that modern LLMs perform at levels similar to or better than supervised models in binary actionability detection, though supervised baselines still excel in fine-grained extraction. The study emphasizes transitions of care and ensures patient safety after discharge.