Multimodal LLMs as CEOs: Visual Inputs Boost Reasoning but Hinder Resource Allocation

The newly introduced benchmark, C-SUITE-BENCH, assesses nine advanced large language models acting as CEOs in a series of controlled business decision-making scenarios. It encompasses five decision-making tasks under both paired text-only and multimodal settings across 50 different scenarios. Findings indicate that multimodal inputs enhance evidence-based reasoning, particularly excelling in risk forecasting and board justification. Nevertheless, a paradox arises with multimodal integration: the inclusion of visual business data negatively impacts resource allocation for all nine models, despite improvements in visual grounding. This research, published on arXiv (2608.05864), underscores the strengths and weaknesses of multimodal LLMs in leadership roles, revealing that while visual information can boost certain reasoning capabilities, it may impede tasks that demand accurate resource distribution.