Matrix-Space Reinforcement Learning for Reusing Local Transition Geometry

A novel approach known as Matrix-Space Reinforcement Learning (MSRL) has been introduced to enhance compositional generalization in sequential decision-making tasks. This method utilizes positive semidefinite matrix descriptors to represent segments of trajectories, which consolidate first- and second-order statistics from lifted one-step transitions. By employing this geometric abstraction, MSRL uncovers shared hidden structures, facilitates algebraic composition within an abstract matrix space, and identifies potential transfer opportunities. The study confirms that these descriptors are well-defined, complete for the low-order additive signal class, additive during valid segment compositions, and minimally sufficient among permissible additive descriptors. Additionally, the research indicates that conditioning value functions on trajectory segments can improve learning, addressing the shortcomings of current techniques that fail to consider local transition dynamics.