Healthcare LLM Benchmarks Rely on Untestable Assumptions

A new arXiv paper argues that benchmarks for evaluating large language models (LLMs) in healthcare are insufficient for predicting real-world deployment performance. The authors identify the evaluation-deployment gap as stemming from implicit assumptions about user-model interactions that benchmarks cannot surface. They classify assumptions into two categories: task assumptions, testable from conversation data alone, and outcome assumptions, which require outcome data and behavioral studies. Outcome assumptions depend on human behavior, which benchmarks cannot directly observe. A retrospective analysis of a healthcare randomized controlled trial (RCT) showed the gap splits roughly equally into task and outcome gaps. To address this, the authors propose a framework called Benchm (likely a typo for a new benchmark or methodology). The paper is available on arXiv under ID 2605.22612.