LLMs Show Judgment-Consequence Gap in Healthcare Moral Reasoning

A recent research paper on arXiv (2608.05583) investigates how large language models (LLMs) assess moral responsibility in healthcare, particularly regarding the distribution of limited medical resources. This cross-posted study explores whether LLMs' evaluations align with human perspectives on patient accountability for health-damaging actions and if these evaluations impact resource distribution. The research analyzes various LLMs from different families and capabilities using clinical scenarios derived from previous studies. A significant discovery is the 'judgment-consequence gap': while LLMs generally concur with humans on patient responsibility, they tend to ignore this judgment when allocating resources, opting for random distribution instead of favoring less responsible patients. This gap presents a substantial challenge for implementing LLMs in critical areas like healthcare, where moral reasoning should reflect human values. The findings underscore the necessity of reconciling moral judgments with resultant actions in AI systems for medical decision-making.