MedUP: A New Medical Vision-Language Model Unifying Perception and Understanding

A team of researchers has unveiled MedUP, an innovative Medical Vision-Language Model (Med-VLM) that integrates perception and comprehension within a common token framework. This model tackles issues related to accurate visual perception, segmentation, and grounding that current Med-VLMs encounter. Conventional methods typically express regions as coordinate strings or depend on external modules, leading to gaps in region-language alignment. The standout feature of MedUP is UniMedTok, a region tokenizer that transforms masks into discrete tokens within the LLM vocabulary, enabling seamless integration of mask tokens with text. The researchers compiled UniMed-Train, a substantial dataset of 1.84 million instances encompassing text-guided segmentation, region-grounded understanding, medical VQA, and chain-of-thought (CoT) segmentation. They also developed UniMed-Bench for comprehensive evaluation. Rigorous testing shows that MedUP surpasses native, agentic, and dual-decoder Med-VLMs across all metrics. The paper can be found on arXiv with the identifier 2608.10635.