LLM Feedback Boosts Manuscript Revision Rates by 12.55% in Global Experiment
A comprehensive randomized field study evaluated the effectiveness of large language models (LLMs) in delivering scientific feedback. This research encompassed over 31,000 arXiv preprints from 150 disciplines and included participation from more than 45,000 researchers across 133 regions worldwide. Authors who obtained tailored feedback generated by LLMs experienced a 12.55% relative rise in the likelihood of revising their manuscripts compared to the control group. Additionally, the use of AI feedback led to an increased adoption of LLM tools in authors' future publications, indicating potential long-term changes in scientific methodologies. The global experiment was published on arXiv.
Key facts
- Global large-scale randomized field experiment
- Over 31,000 arXiv preprints used
- 150 fields covered
- More than 45,000 researchers from 133 geographic regions
- 12.55% relative increase in revision rate
- Increased subsequent use of LLM tools
- Study published on arXiv
- Customized LLM-generated feedback delivered
Entities
Institutions
- arXiv