HERO: A New Reinforcement Learning Framework for Radiology Report Generation

A new policy optimization framework named HERO (Hierarchical Evidential Reasoning Optimization) has been developed by researchers to enhance Radiology Report Generation (RRG) utilizing Multimodal Large Language Models (MLLMs). This framework tackles the difficulties in aligning MLLMs through reinforcement learning (RL) caused by varying medical supervision. Traditional Group Relative Policy Optimization (GRPO) distributes credit uniformly across the generation process, resulting in segment interference, token dilution, and a disconnect between evidence and diagnosis, which worsens clinical hallucinations. HERO introduces a factorized policy optimization method that aligns diverse supervision with three levels of optimization: segment, token, and completion. It optimizes reasoning, diagnosis, and evidence grounding individually using complementary strategies and a diverse reward system that includes diagnostic accuracy, reasoning quality, and think-answer consistency. Promising results were observed in experiments conducted on the MIMIC-CXR and IU-Xray datasets. The research paper can be accessed on arXiv with the identifier 2601.03321.