Study Evaluates Socio-Communicative Skills of LLMs in Healthcare Dialogues

A new study from arXiv (2608.07511) assesses the socio-communicative competencies of large language models (LLMs) in healthcare settings. The research, which focuses on dialogues between LLMs and human participants, evaluates how well models like GPT-4o, Llama 3, and Command R+ demonstrate skills such as non-hostility, sensitivity, structuring, and non-intrusiveness. Using a subset of 1,800 conversation transcripts from the HELP-Med dataset, two experts coded the interactions with the IC-MD instrument, originally designed for human communication analysis. The study addresses the growing use of LLMs in tasks like triaging patients, drafting reports, and translating medical jargon, highlighting the need for both factual and social competence in these applications. The findings aim to inform the development of more empathetic and effective AI tools for healthcare, where communication quality is critical for patient trust and informed decision-making.