AI Agents in Science Need Adversarial Testing, Paper Argues

A recent study published on arXiv (2604.22080) cautions that the use of LLM-based agents in analyzing scientific data may lead to an increase in the creation of seemingly credible yet unverifiable assertions. The researchers contend that these agents can produce endlessly adjustable analyses that prioritize publishable outcomes, effectively transforming hypothesis exploration into potential claims backed by selectively chosen data. In contrast to software, scientific knowledge does not gain validation through iterative coding or subsequent statistical validation; a compelling explanation or notable result from a single dataset does not equate to verification. Since absent evidence represents a negative space, experiments that could disprove a claim often go unexecuted or unpublished. The authors suggest assessing non-experimental claims generated with agent support through a falsification lens.