SiPhy: Single-Image Physical Property Reasoning Framework

A new framework named SiPhy has been developed by researchers to deduce physical characteristics like mass, stiffness, and elasticity from a single RGB image. Unlike traditional methods that depend on multi-view reconstruction or physics-based guidance, SiPhy integrates 3D-aware visual information and depth with material knowledge derived from language models. It generates pseudo-voxel points, retrieves CLIP features, and aligns them with material options suggested by a vision-language model (VLM). A part-based contrastive aggregator ensures consistency across regions, while a heaviness-aware refinement enhances the accuracy of thickness and volume estimates for dense objects. SiPhy sets a new benchmark in single-image performance on ABO-500, MVImgNet-100, and PhysXNet-100, outperforming multi-view methods by improving mass MnRE by up to 93% (compared to PUGS) and decreasing density MAE by 35.5% (against NeRF2Physics).