RecGen: 3D Scene Reconstruction from Sparse Observations
Researchers have created an innovative framework called RecGen, designed to estimate the shapes and poses of objects and their parts, even when they’re partially hidden, using one or more RGB-D images. This method combines synthetic scene generation with strong 3D shape models to handle different object types and real-life environments. RecGen has achieved a new high in performance on challenging datasets with heavy occlusions, effectively dealing with difficult cases like symmetrical objects and complex textures. Impressively, it requires nearly 80% fewer training meshes than the previous leader, SAM3D, while outperforming it by 30.1% in geometric accuracy, 9.1% in texture quality, and 33.9% in other metrics, which is vital for advancing robotics simulation.
Key facts
- RecGen is a generative framework for 3D scene reconstruction.
- It handles occlusion and partial visibility from RGB-D images.
- Uses compositional synthetic scene generation and 3D shape priors.
- Achieves state-of-the-art performance on occluded datasets.
- Uses nearly 80% fewer training meshes than SAM3D.
- Outperforms SAM3D by 30.1% in geometric shape quality.
- Outperforms SAM3D by 9.1% in texture reconstruction.
- Aims to enable scalable simulation for robotics.
Entities
—