REVELIO Framework Reveals Failure Modes in Vision-Language Models

A new framework named REVELIO has been developed by researchers to systematically identify interpretable failure modes in Vision-Language Models (VLMs). These models are increasingly utilized in safety-sensitive contexts due to their extensive reasoning and generalization abilities, yet they can encounter severe failures in particular real-world scenarios. REVELIO characterizes a failure mode as a combination of interpretable, relevant concepts—like pedestrian proximity or adverse weather—where a specific VLM consistently fails. To tackle the challenge of navigating a vast discrete combinatorial space, REVELIO integrates two search methods: a diversity-aware beam search for mapping the failure landscape and a Gaussian-process Thompson Sampling approach for wider exploration. This framework seeks to enhance VLM reliability in critical applications, as detailed in a paper on arXiv (2605.12674).