MM-R2: Agentic Planning for Multimodal Retrieval
A new framework called MM-R2 addresses limitations in multimodal retrieval-augmented generation (mRAG) by introducing agentic planning before retrieval. Existing mRAG systems struggle with under-specified retrieval targets and weakly structured search spaces. MM-R2 constructs an intent-grounded retrieval state from image-question pairs and retrieves over a structured KnowledgeMap, where the agent selects relevant units before issuing queries. The work is detailed in arXiv paper 2607.22643.
Key facts
- MM-R2 is a multimodal agentic retrieval framework.
- It reasons before retrieval by modeling what to retrieve and where to search.
- It constructs an intent-grounded retrieval state from the image-question pair.
- Retrieval occurs over a structured KnowledgeMap.
- The agent selects relevant retrieval units before issuing queries.
- The paper is on arXiv with ID 2607.22643.
- Existing mRAG systems retrieve directly from raw multimodal input over a flat evidence space.
- Two key challenges: under-specified retrieval target and weakly structured search space.
Entities
Institutions
- arXiv