Ego2World: Turning Cooking Videos into Executable Worlds for AI Planning
Ego2World has been launched by researchers as a benchmark that transforms egocentric cooking videos into symbolic worlds suitable for belief-state planning. Utilizing the HD-EPIC dataset, it extracts transition rules from video annotations and implements them within a concealed symbolic world graph. The agent relies solely on local observations and execution feedback for planning, effectively addressing the sim-to-real gap in embodied AI. This research confronts the challenges of partial observability in home settings, where agents need to monitor objects and changes in state. The paper can be accessed on arXiv.
Key facts
- Ego2World converts egocentric cooking videos into executable symbolic worlds.
- It is built on the HD-EPIC dataset.
- The benchmark uses graph-transition rules derived from video annotations.
- The simulator maintains a hidden world graph during evaluation.
- The agent plans over its own partial belief graph using local observations and execution feedback.
- The work addresses the sim-to-real gap in embodied AI.
- It focuses on belief-state planning under partial observation.
- The paper is available on arXiv.
Entities
Institutions
- arXiv