VLM3: Vision Language Models as Native 3D Learners

A recent study, titled 'VLM3: Vision Language Models Are Native 3D Learners,' posits that Vision Language Models (VLMs) possess an innate ability for 3D comprehension without the need for intricate task-specific configurations. Published on arXiv (2605.30561), the research highlights three essential elements—focal length unification, text-based pixel reference, and data mixture and scaling—as adequate for proficient 3D learning. The authors assert that modifications to model architecture, the use of large models, extensive data augmentations, and complicated losses such as regression formulation are unnecessary. They introduce VLM3, a scalable approach that allows standard VLMs to excel in various 3D tasks, thereby significantly improving depth estimation accuracy.