Survey Explores LLM Integration in Peer Review Workflow
A new survey from arXiv examines how large language models (LLMs) can assist or automate stages of the peer review process, including review generation, rebuttals, meta-reviews, and manuscript revisions. The paper categorizes techniques such as fine-tuning, agent-based systems, reinforcement learning, and emerging paradigms. It also covers evaluation methods—human-centered, reference-based, LLM-based, and aspect-oriented—and catalogs datasets and modeling choices. Limitations, ethical concerns, and future directions are discussed. The survey aims to provide practical guidance for building and evaluating LLM systems across the full peer review pipeline.
Key facts
- Survey synthesizes techniques for peer review generation, after-review tasks, and evaluation methods.
- Covers fine-tuning strategies, agent-based systems, RL-based methods for review generation.
- After-review tasks include rebuttals, meta-review, and revision aligned to reviews.
- Evaluation methods span human-centered, reference-based, LLM-based, and aspect-oriented approaches.
- Catalogs datasets and compares modeling choices.
- Discusses limitations, ethical concerns, and future directions.
- Aims to provide practical guidance for building, evaluating, and integrating LLM systems.
- Published on arXiv under Computer Science > Computation and Language.
Entities
Institutions
- arXiv