LLM Reliability Audit for Psychiatric Hospitalization Risk Scores
A recent study introduces a structured method for evaluating the dependability of large language models (LLMs) in clinical applications, focusing on predicting hospitalization risk within psychiatry. This research, available on arXiv (2604.22063), tackles issues related to algorithmic bias and sensitivity to prompts in LLMs employed for clinical decision-making. The researchers create a set of 50 synthetic patient profiles, each incorporating 15 significant clinical features and up to 50 irrelevant ones, to analyze the impact of prompt design and non-medical inputs on hospitalization risk predictions. The goal of this study is to establish a framework for assessing the reliability of LLMs in crucial psychiatric areas where interpretative reliability is uncertain.
Key facts
- arXiv paper 2604.22063 proposes reliability auditing for downstream LLM tasks in psychiatry.
- Focuses on LLM-generated hospitalization risk scores as the first downstream AI clinical-decision-making task.
- Uses a cohort of 50 synthetic patient profiles with 15 clinically relevant and up to 50 clinically insignificant features.
- Evaluates impact of prompt design and inclusion of medically insignificant inputs on risk scores.
- Addresses prior work on algorithmic biases and prompt sensitivity in LLMs.
Entities
Institutions
- arXiv