New Dataset and Model for Scientific Figure Assessment in Peer Review
A new method called SciFigAlign has been developed by researchers to assess scientific figures during peer review, tackling the shortcomings of conventional image quality evaluation techniques. An annotated dataset comprising 3,857 figures from peer-reviewed conference papers was compiled, with each figure assessed on four criteria: Clarity, Relevance, Informativeness, and one additional unspecified criterion. The research indicates that traditional IQA models are inadequate for evaluating the support of scientific arguments, while CLIP-based approaches miss the manuscript's context, and zero-shot LLM/VLM assessments yield excessively focused scores. This initiative seeks to enhance automated scoring of figures by integrating visual and textual information.
Key facts
- SciFigAlign is a new method for scoring scientific figures.
- Dataset contains 3,857 figures from peer-reviewed conference papers.
- Figures rated on Clarity, Relevance, Informativeness, and a fourth dimension.
- Traditional IQA models fail to assess scientific argument support.
- CLIP-based methods lack understanding of manuscript context.
- Zero-shot LLM/VLM judges yield overly concentrated scores.
- The approach focuses on aligning visual and textual evidence.
- Published on arXiv with ID 2607.27066.
Entities
Institutions
- arXiv