‘Will AI Kill Us All?’ Is the Wrong Question to Ask | Analysis

When it comes to artificial intelligence, the debate over whether AI “will kill us all” is less relevant than asking why it’s taking so long for someone to do something about it.

The notion that AI poses an existential threat isn’t new — “Terminator 2” called it back in 1991 — but warnings from some of the people working on these AI systems have dragged this threat into reality. The latest red flag was raised on Tuesday, when Anthropic AI researcher Jacob Coxon said on X that he had resigned from the company and claimed “the people building AI earnestly believe that it could kill us all by the end of the decade.”

While raising the potential existential threat of AI is laudable, it misses the point of what this ecosystem really needs: oversight. For all the talk of AI regulations and safeguards needed in financial services, healthcare and, of course, in Hollywood, there’s less discussion about oversight for the companies building the AI models themselves.

“We spend enormous amounts of time discussing how enterprises should govern AI, yet very little time asking how society should govern the organizations creating the AI itself,” Frost & Sullivan analyst Dilip Sarangan said on a post on LinkedIn. “When did society become a testing environment?”

And while lawmakers like Sen. Bernie Sanders (I-Vermont) are waking up to this danger, they face an uphill battle getting any regulations passed. That’s because the biggest tech giants are caught in a frantic race to build more and more powerful systems, all in the hopes of getting to an end state called superintelligence, where AI is self aware and cognitively more capable than a human. The alternative is letting a competitor — or a company out of China — win the race. It doesn’t help that tech companies have largely cozied up to the Trump administration through ballroom donations and gifts, with Trump himself gung-ho about America’s leadership in AI.

It’s why companies are pouring hundreds of billions of dollars a year into these projects, and why small AI startups and key talent can fetch massive amounts of money. Just last week, Hugging Face, an open-source AI platform and community, agreed to be acquired by chip giant Nvidia for $12.9 billion. That FOMO is driving a winding game with world-dominating — or ending — stakes, and we’re stuck on the ride.

“The arms race for AI has thrown judgment out the window,” said Maribel Lopez, a tech analyst for Lopez Research. “It’s particularly bad in an environment where no one wants to be left behind so they throw safeguards out the window first in the name of progress.”

Coxon said as much in his post: “A common response is ‘if they truly believe this, why are they still building it?’ At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk.”