LLM Use and Scientific Productivity: Robust Association Beyond Stopping-Time Bias
A new arXiv preprint (2607.28968) challenges the claim that the observed positive association between large language model (LLM) adoption and scientific productivity is merely an artifact of stopping-time selection. The authors, responding to Renault, Bergeaud, and Bosquet (RBB), argue that while RBB's mechanism is mathematically possible, it does not prove a null effect. By recalibrating RBB's random placebo to the detector's realized flag rate, they show the measured association remains well above the benchmark, indicating the artifact is too small to explain productivity changes. They further re-estimate the association using complementary designs immune to timing bias: a before-and-after comparison dating adoption in one year and measuring output in another, a conservative control group for difference-in-differences, and an intensity-based specification that never defines an adoption date. The findings reinforce a robust link between LLM use and scientific output, with implications for research evaluation and policy.
Key facts
- The preprint is arXiv:2607.28968v1.
- It responds to Renault, Bergeaud, and Bosquet (RBB) on LLM adoption and productivity.
- RBB argued that dating LLM adoption by first flagged abstract induces stopping-time selection.
- The authors recalibrate RBB's random placebo to the detector's realized flag rate.
- The measured association stays well above the recalibrated benchmark.
- They use three complementary designs to avoid timing bias.
- The designs include before-and-after, conservative control group, and intensity-based specifications.
- The results support a robust positive association between LLM use and productivity.
Entities
Institutions
- arXiv