Study Reveals LLM Judges Perform Poorly in Medical Response Evaluation

A recent study raises doubts about the effectiveness of LLM-as-a-Judge systems in critical medical situations, revealing that their performance is nearly at random when assessing patient-facing medical responses. The research evaluated three levels of rubric detail—General-Likert, Analytical-Rubric, and Dynamic-Checklist—across three foundational models, utilizing two clinician-annotated datasets, including HealthBench, which is the largest publicly accessible benchmark for evaluating medical responses. The LLM Judges recorded AUC scores ranging from 0.49 to 0.66, showing minimal ability to distinguish between complete and incomplete responses. Clinicians would still need to examine the majority of the dataset to identify 90% of incomplete responses, indicating limited practical use for triage. Additionally, when model and clinician assessments aligned, they seldom referenced the same reasoning. Divergent verdicts frequently led to false positives due to the over-identification of non-critical details. This research, documented as arXiv:2604.16383v1, highlights that automated assessments cannot substitute for human expert judgment in medical fields, underscoring crucial implications for the implementation of AI in healthcare where precision is vital.