AI researcher Jacob Coxon left Anthropic after only four months, and his departure has intensified debate over the existential risks posed by artificial intelligence. In a post on X, Coxon warned that many people developing AI genuinely believe the technology could pose an existential threat to humanity before the decade ends. His comments were partly echoed by Evan Hubinger, a senior Anthropic researcher, who said he puts the possibility of such an outcome within the next 10 years at over 10%. Those statements have renewed calls for greater oversight of AI development and reignited debate over how much control humans should retain as systems become more capable.
But what could an AI-driven catastrophe actually look like? Most theories center on hypothetical super-intelligent systems that outperform humans across a wide range of tasks. One argument is that humans may struggle to anticipate the actions of a machine substantially smarter than its creators. Another scenario, famously described by philosopher Nick Bostrom, involves a machine programmed to maximize paperclip production. If given overwhelming capabilities and poorly designed instructions, such a system could theoretically treat everything, including people and natural resources, as material for achieving its objective. The machine would not need to be hostile; it would simply pursue its assigned target without considering consequences that humans regard as important. However, intelligence does not automatically provide unlimited power, and real-world obstacles, laws, institutions, and public resistance could prevent an advanced system from carrying out such an extreme plan.
Biological threats represent another concern. Forecasting exercises such as AI 2027 have included catastrophic pandemics among possible outcomes. Recent research has also demonstrated how AI models can assist with designing genetic sequences, raising questions about how easily advanced technology could lower barriers to biological experimentation. Yet eliminating humanity with a pathogen would be extremely difficult, as a disease must spread efficiently to reach large populations, while highly lethal infections can face biological limits that restrict transmission.
AI could also increase nuclear risks if it were able to interfere with military systems or generate misleading information during a crisis. Cyberattacks have already shown that supposedly isolated infrastructure can be vulnerable, as the Stuxnet attack on Iranian nuclear facilities demonstrated. Although global nuclear arsenals have declined from their Cold War peak, existing weapons could still cause enormous casualties and potentially trigger widespread famine through climate effects associated with a nuclear winter.
The final scenario does not require machines to directly attack humanity. AI could instead contribute to social instability through widespread unemployment, misinformation, political division, and increasingly convincing artificial relationships. If those pressures seriously weakened institutions and communities, the resulting damage could become difficult to contain.
These possibilities illustrate genuine areas of concern, but they remain hypothetical to varying degrees. The challenge is to prepare for credible risks without treating every worst-case scenario as inevitable. The risks should not distract humanity from the immense benefits that AI is helping many industries and businesses like Datavault Inc. (NASDAQ: DVLT) to actualize as they serve their niche of clients.
