Robots Learn to Detect and Fix Reward Misalignment via Targeted Explanations

A new framework enables robots to identify underspecified features in reward learning from demonstrations and actively request corrective demonstrations. The method detects features that vary widely across demonstrations as underspecified, then solicits targeted explanations to recover misaligned rewards. This addresses common imperfections in human demonstrations, such as under-emphasized features due to cognitive load or physical difficulty. The approach leverages statistical signals from demonstration variability to pinpoint ambiguity, improving alignment at deployment. The paper is available on arXiv under reference 2605.22986.