Offline RL for Sepsis Hemodynamic Management: MIMIC-IV Study with Dual Off-Policy Evaluation

A new study from arXiv (2608.16482) applies offline reinforcement learning to optimize intravenous fluid and vasopressor dosing for sepsis patients in the ICU. The research models clinical decisions as a Markov decision process with 1,000 states and 25 actions, using a cohort of 36,872 septic ICU stays from the MIMIC-IV database. The authors employ policy iteration and estimate the clinicians' behavior policy with a random forest. To address the fragility of off-policy estimates, they combine off-policy estimation with reliability diagnostics and clinician-agreement analyses in a transparent validation framework. The study aims to improve the reliable evaluation of sepsis treatment policies, acknowledging that learned policies cannot be trialed on patients directly. This work contributes to the growing field of AI in critical care, offering a methodology for validating treatment recommendations before clinical deployment.