RealityBridge: New Framework to Reduce Sim-to-Real Gap in 3DGS Driving Simulations
A new framework named RealityBridge has been developed by researchers to tackle the considerable Sim-to-Real gap in editable 3D Gaussian Splatting (3DGS) driving simulations. Collecting long-tail hazardous scenarios, essential for safety-focused autonomous driving, is challenging at scale. While editable 3DGS simulations provide a scalable solution through real-scene reconstruction and adjustable editing, the resulting videos often suffer from rendering artifacts, poor foreground quality, lighting inconsistencies, and temporal flickering. Current methods usually only fix a portion of these intertwined issues. RealityBridge seeks to restore local appearance, harmonize edited elements, and ensure temporal consistency, transforming edited 3DGS outputs into realistic driving videos while retaining the defined structure, edits, and dynamics of the simulator. This approach is elaborated in the arXiv paper (arXiv:2606.16278v3) and addresses a vital challenge in autonomous driving simulation, potentially enhancing the safety of testing autonomous vehicles in rare, hazardous situations.
Key facts
- RealityBridge is a video restoration and harmonization framework.
- It addresses the Sim-to-Real gap in editable 3D Gaussian Splatting (3DGS) driving simulations.
- Long-tail hazardous scenarios are difficult to collect at scale for autonomous driving.
- Editable 3DGS simulation offers a scalable alternative via real-scene reconstruction and controllable editing.
- Edited 3DGS-rendered videos exhibit rendering artifacts, degraded foreground assets, illumination mismatch, and temporal flickering.
- Existing methods address only a subset of the coupled defects.
- RealityBridge jointly restores local appearance, harmonizes edited content, and maintains temporal consistency.
- The paper is available on arXiv with identifier 2606.16278v3.
Entities
Institutions
- arXiv