MARA Framework Introduces Query-Adaptive Mechanisms for Multimodal Document Question Answering

The Multimodal Adaptive Retrieval-Augmented (MARA) framework tackles shortcomings in multimodal document question answering that relies on retrieval. Existing methods utilize query-agnostic document representations, overlooking important content and relying on static top-k evidence selection, which does not adapt well to uncertain information distributions. MARA introduces mechanisms that adapt to queries for both retrieval and generation. It features a Query-Aligned Region Encoder that creates multi-level document representations and adjusts them based on their relevance to the query, enhancing retrieval accuracy. Additionally, the framework incorporates a Self-Re... (truncated in source). This study was published on arXiv under the identifier 2604.16313v1 as a cross announcement. Retrieval-based multimodal document QA seeks to extract and combine pertinent information from complex, visually rich documents. Although retrieval-augmented generation (RAG) has excelled in text-based QA, its application to multimodal documents is still largely unexamined.