Language Prompts Recode Visual Representations in Vision-Language Models
A new paper on arXiv (2608.00035) investigates how vision-language models (VLMs) recode visual representations when given goal-directed language. The authors identify an abstract reference representation that denotes goal-relevant objects under natural language prompts, and extract contrastive steering vectors that are causally implicated in model predictions. The work suggests that visual representations in VLMs are not static but can be dynamically recoded based on linguistic context. The paper is available at https://arxiv.org/abs/2608.00035.
Key facts
- Paper on arXiv with ID 2608.00035
- Focuses on vision-language models (VLMs)
- Identifies an abstract reference representation for goal-relevant objects
- Uses contrastive steering vectors
- Demonstrates causal implication in model predictions
- Provides evidence for language-induced recoding of visual representations
- Published as new announcement type
- Available at https://arxiv.org/abs/2608.00035
Entities
Institutions
- arXiv