LLMs Fail Pluralistic Reasoning: New Study Proposes Deliberative Reason Index

A new paper on arXiv (2608.10186) critiques the deployment of large language models (LLMs) in settings requiring collective reasoning on complex, value-laden problems. The authors argue that current benchmarks, which focus on verifiable tasks like mathematics and coding, do not adequately assess LLM performance on problems without objectively correct answers, where decision quality depends on integrating pluralistic perspectives. They contend that procedural evaluations of LLM discourse—such as respectfulness, justification, and engagement—are systematically insufficient. To address this, they apply the Deliberative Reason Index (DRI), a measure from political science validated across citizen assemblies, to evaluate reliable group-level reasoning on pluralistic, non-verifiable problems. The paper synthesizes recent evidence to support their argument. The study highlights a growing concern in AI ethics and governance: as LLMs are increasingly used in public discourse and decision-making, their ability to handle value-laden, pluralistic issues remains under-tested. The authors propose DRI as a tool for more robust evaluation, potentially influencing future AI deployment policies.