OpenAI's Math Proofs Fall Short of AGMAI Standards, Cambridge and King's College Paper Finds

This week, OpenAI unveiled hundreds of purported solutions to some of the most challenging mathematical problems globally, having consulted a group of elite mathematicians to prevent controversy following a previous model's resolution of a longstanding issue. The Advisory Group on Mathematics and Artificial Intelligence (AGMAI), which consists of nine distinguished researchers and is hosted by Princeton University's Institute for Advanced Studies, provided guidelines at the end of September. AGMAI's initial recommendation was to refrain from testing advanced mathematical problems using proprietary models; however, OpenAI's release assesses proprietary models against open research challenges. Only 10 out of 719 manuscripts presented the model's reasoning, and merely 42% of proofs had not been formalized. A recent study from mathematicians at the University of Cambridge and King's College in London identified at least two inconsistencies between the natural language proof and the Lean code for OpenAI's solution to a Navier-Stokes equations problem. AGMAI requested that OpenAI incorporate machine-readable metadata linking natural language with formal artifacts, which was not fulfilled. Terence Tao criticized AI prompters for their lack of interest in the broader field, while Harvard mathematics professor Melanie Wood remarked to TechCrunch that there is no genuine human understanding at the time of release.