45 scientists rate AI reviewers in Nature-family paper evaluation study

A recent research paper available on arXiv (2605.20668) investigates the role of AI reviewers in the scientific peer review process through extensive expert evaluation. In total, 45 specialists from the fields of Physical, Biological, and Health Sciences dedicated 469 hours to assessing 2,960 distinct critiques generated by AI reviewers of Nature-family publications. The objective of this study is to delineate the strengths and weaknesses of AI reviewers, moving beyond mere agreement with human evaluations. It contributes to the ongoing discussion between those who are skeptical of AI's expertise and those who believe in its potential despite a lack of supporting evidence.