TimeRewarder: Learning Dense Reward from Passive Videos via Temporal Distance

TimeRewarder introduces an innovative approach to reward learning by extracting progress estimation signals from passive video sources, such as human videos and robot demonstrations, through the analysis of temporal distances between pairs of frames. This method provides incremental proxy rewards to facilitate reinforcement learning. In tests involving ten Meta-World tasks, TimeRewarder significantly enhances performance in sparse-reward scenarios, attaining nearly flawless success in 9 out of 10 tasks with merely 200,000 interactions per task, surpassing earlier techniques.