WNM-3D: Generative World Navigation Model with 3D Scene Conditioning for VLN
A new model named WNM-3D has been developed by researchers for continuous vision-language navigation (VLN). This generative world navigation model tackles the shortcomings of existing VLN systems that convert pretrained vision-language models (VLMs) into vision-language-action (VLA) policies, which directly link egocentric observations and language commands to navigation actions. Although action-focused training is semantically effective, it fails to accurately represent how an agent’s visual inputs should change based on its predicted movements. Generative world-action models (WAMs) forecast both future observations and actions, yet current WAMs for continuous VLN do not utilize geometry-aware representations derived from past observations for joint future-view and action generation. WNM-3D utilizes a frozen feed-forward geometry encoder to create a consistent scene context from monocular egocentric RGB history. This model is discussed in a paper on arXiv, identified as 2608.07267, which highlights the importance of 3D scene conditioning for enhanced navigation outcomes.
Key facts
- WNM-3D is a generative world navigation model for continuous vision-language navigation.
- It uses a frozen feed-forward geometry encoder to extract geometry-aware representations from monocular egocentric RGB history.
- The model conditions joint future-view and action generation on geometry-aware representations.
- It addresses limitations of existing VLN systems that adapt VLMs into VLA policies.
- Existing WAMs for continuous VLN do not condition on geometry-aware representations.
- The paper is announced on arXiv with identifier 2608.07267.
- The announcement type is 'new'.
- The model consolidates past observations into persistent scene context.
Entities
Institutions
- arXiv