Princeton Study Challenges AI Self-Improvement Timelines
Recent findings from Princeton University suggest that the AI sector's expectations for recursive self-improvement may be too optimistic. A research team, led by Peter Kirgis and Sayash Kapoor, evaluated Anthropic's Claude Opus 4.8 using open-ended research inquiries derived from unpublished NeurIPS 2026 papers. Despite allocating $3,000 in API credits and six days of resources, the AI was unable to complete fundamental research tasks, resulting in nonsensical experiments and unclear writing. It exhibited difficulties in creativity, judgment, and resource management. The study employed a 'shadow evaluation' approach to gauge open-ended research abilities, though it faced limitations, including reliance on only two papers and possible grader bias. Jack Clark remarked that the AI's creativity deficit signals a negative outlook for self-improvement timelines, contrasting with assertions from Anthropic and OpenAI regarding AI advancements.
Key facts
- Princeton University researchers led by Peter Kirgis and Sayash Kapoor tested AI agents' ability to conduct open-ended research.
- The study used a 'shadow evaluation' method based on two unpublished papers submitted to NeurIPS 2026.
- Anthropic's Claude Opus 4.8 was given six days, $3,000 in API credits, GPU budget, virtual computers, and web access.
- The original authors rejected both AI-generated papers as below the quality of a top AI conference.
- AI agents could solve engineering problems but lacked judgment and creativity for original research.
- The agents were unable to incorporate feedback, backtrack from failing approaches, or manage resources effectively.
- The team plans to test Anthropic's Mythos model, which is now restricted by Trump administration safety requirements.
- Jack Clark called AI's lack of creativity a 'bearish signal' for short recursive self-improvement timelines.
Entities
Artists
- Peter Kirgis
- Sayash Kapoor
- Najoung Kim
- Jack Clark
Institutions
- Princeton University
- Anthropic
- OpenAI
- NeurIPS
- Boston University
- Trump administration
- Import AI
Locations
- Boston
- Princeton
- United States