Lifted World Models Enable High-Level Planning for Embodied Agents

The recently introduced framework in arXiv preprint 2604.26182 presents lifted world models that translate high-level actions into sequences of low-level joint movements, facilitating efficient planning for complex entities such as human agents. Conventional world models forecast future observations based on actions, but the high-dimensional nature of action spaces (like controlling each joint of a human) leads to poor scalability for search-based techniques like CEM. The innovative approach involves training a lightweight policy that integrates with a static world model, resulting in a lifted model capable of predicting future observations from a single high-level action. For human-like embodiments, the high-level action space consists of a limited set of 2D waypoints marked on the current observation frame, each indicating a near-term target position for a leaf joint (pelvis), thereby simplifying planning and enhancing control scalability.