FairChart2Table Framework Exposes Y-Axis Bias in Multimodal Language Models
A new framework, FairChart2Table, systematically analyzes y-axis-related biases in five state-of-the-art Multimodal Language Models (MLMs) for chart-to-table translation. The study reveals significant biases tied to digit length of major tick values, number of major ticks, value range, and tick format (abbreviation or scientific). The number of legends or entities in chart images also impacts MLM performance. Prompting MLMs with y-axis information significantly enhances accuracy. The research addresses a gap in prior work, which had not systematically examined these biases. The findings underscore the need for balanced datasets to ensure fair MLM performance across diverse chart types.
Key facts
- FairChart2Table framework proposed for analyzing y-axis bias in MLMs.
- Five state-of-the-art MLMs evaluated.
- Biases found in digit length, number of major ticks, value range, and tick format.
- Number of legends/entities affects MLM performance.
- Prompting with y-axis information improves performance.
- Study highlights imbalances in public chart datasets.
- Previous works did not systematically examine y-axis biases.
- Research published on arXiv (2604.24987).
Entities
Institutions
- arXiv