See2Refine Framework Uses Vision-Language Models to Enhance AI-Generated Vehicle Communication

The newly introduced framework, See2Refine, tackles the issue of automated vehicles conveying their intentions to other road users via external Human-Machine Interfaces (eHMIs). Conventional studies on eHMIs often rely on predetermined message-action pairs crafted by developers, which struggle to adapt to the complexities of changing traffic situations. While Large Language Models (LLMs) have shown promise as designers for context-aware eHMI actions, they generally lack perceptual validation and depend on fixed prompts or costly human feedback for improvement. See2Refine presents a human-free, closed-loop system that utilizes a vision-language model (VLM) to provide automated visual feedback, enhancing the LLM-based eHMI action designer. This method evaluates the suitability of proposed eHMI actions in specific driving contexts, refining them without manual input. The framework is discussed in a research paper, arXiv:2602.02063v2, released as a replace-cross type on arXiv. Given that automated vehicles currently lack effective communication methods, eHMIs play a crucial role in expressing intent and ensuring safety. The research underscores the shortcomings of current techniques and positions See2Refine as a scalable solution for adapting to dynamic traffic conditions.