Semi-Markov RL for EV Ride-Hailing with Feasibility Guarantees

A novel semi-Markov reinforcement learning framework has been created for large-scale electric vehicle (EV) ride-hailing operations, ensuring that actions remain physically feasible. This issue is approached as a hex-grid semi-Markov decision process (semi-MDP), incorporating mixed actions that include discrete choices (serving, repositioning, charging) alongside continuous charging power and varying durations. To maintain feasibility throughout training and implementation, the policy is informed by high-level intentions generated by a masked, temperature-annealed actor. These intentions are applied at each decision point via a time-constrained rolling mixed-integer linear program (MILP) that enforces state-of-charge, port, and feeder limitations. To address distributional shifts, a Soft Actor-Critic (SAC) agent is optimized against a Wasserstein-1 ambiguity set with a graph-based framework. This research tackles the complexities of uncertain, spatially correlated demand and travel times while adhering to charger and feeder constraints. The study can be accessed on arXiv with ID 2604.25848.