Study Finds LLMs Induce Harmful Decision Flips in Clinical Settings

A new study from arXiv (2608.14630) investigates how large language models (LLMs) can amplify cognitive biases in human decision-making, particularly in high-stakes clinical contexts. The researchers developed a decision-theoretic framework to analyze 'rhetorical misalignment,' a failure mode where LLMs present information in rhetorically inappropriate ways for a given decision context, leading to suboptimal human decisions. Through a human-subject experiment using realistic clinical decision-making scenarios derived from the United States Medical Licensing Examination, the study measured the impact of LLM-generated information on decisions. Results showed that LLMs induced an average 2.81% rate of harmful decision flips, where participants changed from correct to incorrect decisions due to LLM output. The study highlights the potential risks of integrating LLMs into decision-making processes, especially in fields like medicine where errors can have severe consequences. The findings underscore the need for careful design and evaluation of AI systems to mitigate such biases and ensure safe human-AI collaboration.