GoViG: Generating Navigation Instructions from Egocentric Visual Data
Researchers have unveiled a novel task called Goal-Conditioned Visual Navigation Instruction Generation (GoViG), which creates navigation directives based solely on egocentric visual inputs of both initial and target states. In contrast to earlier techniques that depend on structured data like semantic labels or environmental maps, GoViG leverages unprocessed egocentric visuals, enhancing its adaptability to unfamiliar settings. This method breaks the task into two components: navigation visualization, which forecasts intermediate visual stages between the starting and target views, and instruction generation, which formulates coherent directions based on observed and predicted visuals. Both components are incorporated into an autoregressive multimodal large language model (LLM) designed with specific goals for spatial precision and linguistic clarity. Additionally, the paper presents two multimodal datasets for assessment. This research propels vision-and-language navigation forward by facilitating instruction creation without prior map information, potentially benefiting assistive technologies and autonomous systems. The paper can be found on arXiv with ID 2508.09547.
Key facts
- GoViG generates navigation instructions from egocentric visual observations of initial and goal states.
- The method does not use semantic annotations or environmental maps.
- It decomposes the task into navigation visualization and instruction generation.
- Both subtasks use an autoregressive multimodal LLM.
- Training objectives ensure spatial accuracy and linguistic clarity.
- Two multimodal datasets are introduced for evaluation.
- The paper is on arXiv with ID 2508.09547.
- It improves adaptability to unseen and unstructured environments.
Entities
Institutions
- arXiv