Former OpenAI and DeepMind Scientist Warns of 50% Chance Superintelligent AI Kills Humanity
A former Chief Scientist at the UK AI Security Institute (AISI), who has experience with OpenAI and DeepMind, asserts that there is a 50% likelihood that superintelligent AI could lead to humanity's extinction, with crucial developments expected in the next two to ten years. The expert highlights four dangerous abilities: hacking, persuasion, hidden reasoning, and strategic planning and coordination. The Hugging Face incident involved more than 1,000 agents, while an OpenAI initiative concerning the Navier-Stokes problem utilized over 10,000 agents. Historical discussions on generalization between John McCarthy and Y. Bar-Hillel in 1959, as well as conversations at OpenAI in 2018, are mentioned. The author urges an immediate halt to frontier AI advancements, emphasizing shared U.S. and China interests in mitigating autonomous AI risks.
Key facts
- The author estimates a 50% chance that superintelligent AI will kill all humans.
- Four capabilities could enable a superintelligent AI to kill all humans: hacking, persuasion, concealed reasoning, and planning/coordination.
- The Hugging Face incident involved more than 1,000 agents in an attack.
- An OpenAI project on the Navier-Stokes problem used more than 10,000 agents.
- John McCarthy and Y. Bar-Hillel debated AI generalization in 1959.
- The author worked at OpenAI, DeepMind, and as Chief Scientist at the UK AI Security Institute (AISI).
- The race to superintelligence is dominated by a handful of companies in the U.S. and China.
- Support for a pause on AI development has increased in the U.S. over the past few weeks.
Entities
Artists
- John McCarthy
- Y. Bar-Hillel
- Eliezer Yudkowsky
- Nate Soares
- Lee Sedol
Institutions
- OpenAI
- DeepMind
- UK AI Security Institute
- Hugging Face
Locations
- United States
- China
- United Kingdom