Argus-Unified: Compact AI Model for Image Understanding and Generation
A new AI model named Argus-Unified has been unveiled by researchers, designed to integrate visual comprehension and creation while minimizing computational and data needs. This compact multimodal model utilizes pretrained vision-language models (VLMs) to establish robust multimodal foundations, employing hybrid visual tokens: continuous tokens for comprehension and discrete tokens for generation, all relying on a fixed unified vision encoder. Its training process consists of two phases: the initial phase focuses on developing a quantizer and image decoder atop the frozen encoder, followed by training the LLM, which is initialized from a pretrained VLM, to handle unified multimodal tasks. This strategy effectively tackles the challenges posed by high compute and data demands, as well as the conflicts between visual features for understanding and generation.
Key facts
- Argus-Unified is a compact multimodal model for image understanding and generation.
- It uses pretrained vision-language models (VLMs) for strong multimodal priors.
- Hybrid visual tokens include continuous tokens for understanding and discrete tokens for generation.
- The vision encoder is frozen and unified.
- Training pipeline has two stages: quantizer/image decoder learning, then LLM training.
- The model aims to reduce compute and data demands.
- It addresses conflicts between visual features for understanding and generation.
- The paper is available on arXiv (2607.25527).
Entities
Institutions
- arXiv