Generative Framework Creates High-Fidelity Simulation Scenes from Real-World Panoramas for Robot Learning
A novel generative framework has been developed to connect real-world environments with high-fidelity simulation scenes, tackling the challenges of cost and scalability in collecting robot data from the real world. This method generates varied cousin scenes through both semantic and geometric modifications, facilitating effective data augmentation for robot policy evaluation and learning. Interactive manipulation tasks are supported by high-quality physics engines and realistic assets within these created environments. Techniques for multi-room stitching enable the assembly of consistent large-scale settings for extended navigation through intricate layouts. Experiments indicate a robust correlation between simulation and reality, confirming the platform's accuracy. This research, identified as arXiv:2604.15805v1, offers a viable solution for enhancing real-world scenes in simulations, addressing the hurdles of acquiring physical assets and reconfiguring environments.
Key facts
- Generative framework maps real-world panoramas to simulation scenes
- Synthesizes diverse cousin scenes via semantic and geometric editing
- Supports interactive manipulation tasks with physics engines
- Uses multi-room stitching for large-scale navigation environments
- Demonstrates strong sim-to-real correlation in experiments
- Addresses costly real-world data collection for robots
- Enables efficient data augmentation for robot learning
- Research announced on arXiv as arXiv:2604.15805v1
Entities
Institutions
- arXiv