RubricReviewer: A Rubric-Driven LLM Framework for Objective Peer Review
RubricReviewer is an innovative framework designed to improve peer review processes in prominent academic settings by employing large language models (LLMs) as assistants. It tackles two key issues faced by existing LLM reviewers: the unspoken correlation between manuscripts and their reviews, and the inadequate representation of high-quality reviews. By explicitly creating rubrics, RubricReviewer tailors review generation and evaluation to criteria that adapt to each paper. This framework integrates a training-free agent, known as Scout, for gathering evidence alongside a training-based element that incorporates human judgment. Introduced in a paper on arXiv (ID: 2608.00005), it seeks to deliver objective, thorough rubric-based peer reviews in response to growing submission demands. The authors, venue, and publication date remain undisclosed.
Key facts
- RubricReviewer is a fully rubric-driven framework for peer review using LLMs.
- It addresses two structural limitations: implicit rubrics and incomplete review paradigms.
- The framework makes rubric generation an explicit intermediate step.
- It combines a training-free agent (Scout) that gathers external evidence.
- The paper is posted on arXiv with ID 2608.00005 and annotation type 'cross'.
- Peer review at major venues is under unprecedented submission pressure.
- Existing LLM-based reviewers face limitations in mapping manuscripts to reviews.
- Training-based reviewers inherit human discriminative judgement with noise and uneven coverage.
Entities
Institutions
- arXiv