Robotic chemistry lab tests AI agents' scientific agency

A recent study released on arXiv investigates the capacity of AI agents to perform scientific tasks within a robotic chemistry lab designed as a real-world testing environment. Featuring 45 modular workstations that provided machine-readable skills, the lab facilitated 4,608 experiments. Merely 3.3% of these trials resulted in workflows deemed executable by experts under the lab's constraints, with the top-performing system achieving 28.1%. Long-term planning proved difficult, as only three workflows surpassed 30 operations, with the longest comprising 44. Throughout five experimental rounds, feedback led to minor adjustments but did not result in any major workflow reconfigurations or redesigns of analytical methods. This study offers a data-driven evaluation of readiness for deployment and a framework for guiding iterative improvements.