SISL: Self-Improving Skill Learning for Robust Meta-Reinforcement Learning

A novel approach known as Self-Improving Skill Learning (SISL) tackles the issue of noisy offline demonstrations within skill-based meta-reinforcement learning. While meta-RL allows for quick adaptation to unfamiliar tasks, it faces difficulties in long-horizon scenarios. Skill-based methods break down state-action sequences into reusable skills through hierarchical decision-making, but they are particularly vulnerable to noise in offline datasets. SISL enhances skill refinement independently through separate high-level and skill improvement policies, incorporating skill prioritization via maximum return relabeling to concentrate updates on relevant task trajectories. This strategy reduces noise impact, resulting in more robust and stable adaptation. Experimental results indicate that SISL consistently surpasses other skill-based meta-RL techniques across various long-horizon tasks.