QR-Structured Thermal Triggers: New Attack Framework for Infrared Vision-Language Models
A new framework called QR-Structured Thermal Triggers (QR-STT) has been developed by researchers to facilitate targeted semantic assaults on infrared vision-language models (IR-VLMs) without requiring prior training. These models, which enhance thermal perception for various applications like open-vocabulary classification, image captioning, and visual question answering, have not been thoroughly evaluated for their resilience against structured thermal disturbances or the consistency of cross-modal semantic alignment. QR-STT optimizes internal modules, each assigned a specific thermal state—cold, neutral, or hot—while maintaining the functional areas of a QR pattern. It employs a three-stage gradient-free method with greedy module-flip refinement to effectively navigate the mixed search space. This research, highlighting vulnerabilities in IR-VLMs, is documented in a paper on arXiv with the identifier 2607.29445, categorized as 'cross'.
Key facts
- QR-STT is a stealthy, training-free, black-box framework for targeted semantic attacks on IR-VLMs.
- IR-VLMs extend thermal perception to open-vocabulary classification, image captioning, and visual question answering.
- QR-STT preserves functional regions of a QR pattern while optimizing its internal modules.
- Each module is assigned a cold, neutral, or hot thermal state.
- The framework jointly searches module topology and rendering parameters: position, scale, rotation, intensity, blur, and roundness.
- A three-stage gradient-free procedure with greedy module-flip refinement handles the mixed discrete and continuous search space.
- The objective promotes alignment with an attacker-selected target.
- The paper is available on arXiv under identifier 2607.29445.
Entities
Institutions
- arXiv