Study Reveals Answer Inertia in Vision-Language Models' Reasoning
A recent paper on arXiv (2604.14888) explores the integration of visual and textual data in vision-language models (VLMs) during reasoning processes. The research examines 18 VLMs, categorized into instruction-tuned and reasoning-trained types. The team assessed confidence levels throughout Chain-of-Thought (CoT) reasoning, analyzed the impact of reasoning corrections, and looked at the role of intermediate steps. The results indicated that models display 'answer inertia,' where initial predictions tend to be reinforced rather than altered. Although reasoning-trained models demonstrate improved correction abilities, their effectiveness varies based on modality conditions, ranging from text-heavy to vision-exclusive environments. Experiments with misleading textual prompts showed that models are persistently swayed by these cues, even when visual information is adequate. This paper, marked as a replace-cross on arXiv, emphasizes the challenges in monitoring modality dependence in VLMs, raising concerns about AI reliability and safety.
Key facts
- The paper is available on arXiv with ID 2604.14888.
- The study analyzes reasoning dynamics in 18 vision-language models.
- Models from two different model families were examined, including instruction-tuned and reasoning-trained variants.
- Researchers tracked confidence over Chain-of-Thought (CoT) reasoning.
- The study found that models exhibit 'answer inertia,' reinforcing early predictions rather than revising them.
- Reasoning-trained models show stronger corrective behavior, but gains depend on modality conditions.
- Controlled interventions with misleading textual cues showed models are consistently influenced by these cues even when visual evidence is sufficient.
- The paper was announced as a replace-cross on arXiv, indicating a revision.
Entities
Institutions
- arXiv