BiomedAP: Dual-Anchor Framework for Robust Medical Vision-Language Adaptation

A team of researchers has introduced BiomedAP, a dual-anchor framework that utilizes vision-informed gated cross-modal fusion to tackle the sensitivity of biomedical Vision-Language Models (VLMs) to variations in prompts. Current adaptation methods tend to optimize visual and textual prompts separately, resulting in inconsistent cross-modal alignment when faced with noisy clinical descriptions. BiomedAP promotes cohesive alignment through gated cross-modal fusion for layer-wise interaction, alongside a dual-anchor constraint that stabilizes prompts towards reliable semantic centroids derived from expert templates and few-shot examples. The primary goal of this framework is to enhance the robustness of few-shot medical diagnoses.