SAM 3D Animal: First Promptable Multi-Animal 3D Reconstruction from a Single Image
Researchers have developed SAM 3D Animal, the first promptable framework capable of reconstructing multiple animals in 3D from a single image. Built on the SMAL+ parametric model, it handles occlusions and crowded scenes using keypoints and masks as flexible prompts. To train the model, the team created Herd3D, a multi-animal 3D dataset with over 5,000 images featuring diverse species, interactions, and occlusion patterns. The framework achieves state-of-the-art results on Animal3D, APTv2, and Animal Kingdom datasets, outperforming both model-based and model-free methods. This work addresses a critical gap in 3D animal reconstruction, which previously focused on single-animal settings.
Key facts
- SAM 3D Animal is the first promptable framework for multi-animal 3D reconstruction from a single image.
- Built on the SMAL+ parametric animal model.
- Supports prompts in the form of keypoints and masks.
- Introduces Herd3D dataset with over 5,000 images.
- Herd3D increases diversity in species, interactions, and occlusion patterns.
- Achieves state-of-the-art on Animal3D, APTv2, and Animal Kingdom datasets.
- Outperforms existing model-based and model-free methods.
- Addresses challenges of species variation, occlusions, and multi-animal scenes.
Entities
—