Strict Subgoal Execution: A New Hierarchical RL Framework

Researchers propose Strict Subgoal Execution (SSE), a graph-based hierarchical reinforcement learning framework designed to tackle long-horizon goal-conditioned tasks. SSE integrates Frontier Experience Replay (FER) to distinguish unreachable subgoals from admissible ones, improving high-level planning efficiency. FER uses failure and partial-success transitions to define a reachability frontier, filtering unreliable subgoals and reducing unnecessary decisions. The framework also includes a decoupled exploration policy to target underexplored goal regions and a path refinement mechanism. This approach addresses limitations of conventional hindsight relabeling in hierarchical and graph-based methods, which often fail to correct subgoal infeasibility. The work is detailed in a paper on arXiv (2506.21039).