C3PO Benchmark Reveals Modality Dominance in Multimodal AI

A novel benchmark named C3PO has been introduced in an arXiv paper (2608.05381) to assess cross-modal composition and counterfactual performance in omnimodal models. It consists of 3,404 samples that include video, audio, image, and text, aimed at evaluating two key skills: information composition (integrating scattered evidence) and counterfactual conflict (addressing intentional contradictions). Developed through an automated pipeline utilizing 25 logically grounded templates, C3PO incorporates a paired IC/CC structure and a four-tier design to diagnose cross-modal reasoning failures effectively. Findings indicate that humans achieve 88.64% accuracy, while the top model, Gemini-3.1-Pro, only attains 73.17%. Notably, 86-95% of failures arise from modality dominance, where models focus on one modality and overlook conflicting evidence. This research underscores a critical limitation in current multimodal large language models (MLLMs) and serves as a diagnostic tool for the AI research community and multimodal system developers.