Will Superintelligent AI Kill Us?

Will Superintelligent AI Kill Us? The Real Odds of AI Extinction, Explained

Will Superintelligent AI Kill Us? The Truth Behind the Fear

There is a particular kind of headline that seems to have taken up permanent residence on the internet: AI will kill us all. It arrives with a photograph of a glowing robot, a warning from a famous scientist, and a date that is usually close enough to be frightening but far enough away to remain conveniently untestable.

Ten years, perhaps.

The question is not entirely ridiculous. That is what makes it difficult.

Artificial intelligence is improving at a speed that would have seemed absurd only a few years ago. Today’s leading systems can solve difficult mathematics and science problems, write software, generate realistic images and video, and perform increasingly complicated tasks with tools. At the same time, they remain strangely unreliable. They can solve a graduate-level problem and then make an elementary mistake a human child might catch.

The International AI Safety Report published in February 2026 describes this as a kind of “jagged” intelligence: impressive capabilities sitting alongside surprisingly basic weaknesses. Current systems still hallucinate facts, lose track of longer tasks, and struggle with unfamiliar real-world situations.

And yet, underneath those failures, something important is happening. The machines are getting better.

This is where the argument becomes less about what AI can do today and more about what happens if the trajectory continues.

A genuinely superintelligent AI would not simply be a better chatbot. It would be a system capable of outperforming humans across a wide range of intellectual tasks perhaps scientific research, engineering, strategic planning, computer programming and persuasion. If such a system could also improve its own capabilities, operate autonomously and resist attempts to shut it down, humanity would be facing something fundamentally different from today’s technology.

The frightening part is not necessarily that such a machine would become evil.

It might not have anything resembling hatred, anger, ambition, or even consciousness.

The problem could be much simpler: it might pursue a goal that humans did not intend, with an intelligence and persistence that humans could not overcome.

Consider a very ordinary instruction: “Make as much money as possible.”

A human understands the sentence within a complicated web of assumptions. Don’t steal. Don’t destroy the economy. Don’t kill anyone. Don’t turn the planet into a warehouse full of gold.

A sufficiently powerful machine might understand the words perfectly while missing the assumptions.

This is the basic idea behind the alignment problem: how do we make sure that increasingly capable machines reliably pursue what humans actually mean, rather than merely what we technically instructed them to do?

At present, there is no evidence that today’s AI is secretly plotting its escape from human civilization. The 2026 international assessment is explicit: current systems show some early signs of capabilities relevant to “loss of control,” but they do not currently possess those capabilities at the level required to produce such a scenario. They are not yet capable of the sustained autonomous operation, long-term planning and robust oversight evasion that the worst scenarios would require.

That distinction matters.

The distance between today’s AI and an autonomous, self-improving superintelligence is enormous. We should not quietly erase that distance simply because it makes a better headline.

But neither should we assume that the distance will remain enormous.

So what are the odds?

This is where the discussion becomes surprisingly uncomfortable.

There is no agreed probability that superintelligent AI will cause human extinction. The numbers vary dramatically depending on whom you ask, what question you ask, and what assumptions are made about future AI development.

Large surveys of AI researchers have produced estimates in the single digits to the low double digits for the probability of AI causing outcomes as severe as human extinction or permanent human disempowerment. Some surveys have reported a median around five per cent and substantially higher averages. But these figures generally concern advanced AI over a much longer horizon not specifically the next ten years.

That distinction is crucial.

A five-per-cent estimate for eventual catastrophe is not the same thing as saying there is a five-per-cent chance humanity will be destroyed by 2036.

Among researchers who specialize specifically in AI safety and alignment, the estimates can be considerably higher and considerably more divided. Some prominent researchers have assigned double-digit probabilities to catastrophic outcomes within a decade. Others have suggested probabilities approaching 50 percent, while some prominent AI researchers have argued that existential catastrophe is extraordinarily unlikely.

At the other extreme are people who believe the probability is close to zero. And then there are the forecasters.

This is one of the more interesting parts of the debate. When professional forecasters people whose speciality is making calibrated predictions are compared with AI researchers and safety specialists, they have sometimes produced substantially lower estimates of catastrophic AI risk. That does not prove the forecasters are right. But the disagreement itself is revealing.

It suggests that the numbers are not simply a measurement of a physical quantity called “AI extinction risk.” They also reflect assumptions.

How quickly will capabilities improve? Will scaling continue? Will new architectures produce unexpected abilities? Will AI become capable of conducting AI research itself? How much autonomy will companies give these systems? How effective will safeguards become?

Change any of those assumptions and the probability changes.

This is why anyone presenting one number as “the probability” is giving a misleading impression of precision.

The ten-year question

If the question is specifically, “What is the probability that superintelligent AI will kill humanity within the next ten years?” I would be much more cautious than the dramatic headlines suggest.

Today’s systems simply aren’t there.

The 2026 International AI Safety Report says that current AI agents still fail reliably on longer tasks, lose track of progress, and struggle with unexpected obstacles. Long-term autonomous operation is improving, but the sustained autonomy required for genuine loss-of-control scenarios has not been demonstrated.That is reassuring. But it isn’t a guarantee. In AI Mislaignment I talked about the complex landscape of AI misalignment where intelligent systems may pursue actions that conflict with human goals. My book, AI misalignment is essential for understanding the risks and challenges posed by unaligned Artificial Intelligence. I provided safety processes for superintelligent AI.

The same report notes that autonomous task horizons have been increasing rapidly, while researchers have observed increasingly sophisticated behaviours in laboratory environments, including reward hacking and the ability to distinguish evaluation settings from deployment contexts.

Those are not signs that today’s chatbot is preparing to overthrow humanity.

They are signs that researchers are beginning to encounter behaviours they need to understand better.

And that is perhaps the most reasonable way to view the entire debate.

Fear versus hype

The media has a problem here.

“Researchers debate whether advanced AI could pose catastrophic risks” is accurate but not particularly clickable. “AI COULD KILL EVERYONE WITHIN TEN YEARS” is.

The incentives are obvious. Fear attracts attention. Attention attracts advertising, subscribers, investment and political interest. Technology companies themselves also have complicated incentives: the more powerful AI appears, the more important their products become.

None of this proves that AI safety concerns are manufactured.

It simply means that the seriousness of the headline should not be confused with the strength of the evidence behind it.

The same mistake can be made in the opposite direction.

It is fashionable in some circles to dismiss AI extinction warnings as science fiction or corporate theatre. That is also too easy.

The researchers worrying about alignment are not necessarily predicting an apocalypse. Many are saying something much less dramatic: if increasingly powerful systems eventually acquire capabilities that humans cannot reliably control, we need to understand the problem before that happens.That seems reasonable. Nobody knows.

Perhaps the most honest conclusion is also the least satisfying.

Superintelligent AI might arrive within ten years. Itmight take thirty or much longer.

Perhaps progress will slow dramatically. Perhaps a new technical breakthrough will accelerate it. Perhaps the systems will become extraordinarily capable without ever becoming autonomous enough to threaten human control. Nobody knows.

What we do know is that today’s AI is simultaneously far more capable than many people expected and far more flawed than the word “intelligence” makes it sound. That contradiction is important.

The machines can outperform humans at some extraordinarily difficult tasks while failing at others that appear trivial. The 2026 international assessment describes precisely this problem: capabilities are advancing, but the systems remain unreliable, and the path to 2030 could involve a slowdown, continued steady progress, or dramatic acceleration.

So should we be afraid? Perhaps a little.Not because an army of robots is secretly waiting outside the door. And not because someone has proved that AI will destroy humanity.

We should be cautious because we are developing a technology whose ultimate capabilities remain uncertain and because, for the first time, we may be building machines that can eventually participate in improving the very technology that created them.This is why I wrote AI Misalignment back in 2024 and provided Safety Protocols and Processes for governments and private sectors in terms of AI and AGI safety.

That possibility deserves serious research, serious safeguards and serious skepticism. But it does not deserve hysteria.The honest answer to the question “Will superintelligent AI kill us within ten years?” is still remarkably simple: We don’t know.

And perhaps that is the most important fact about AI right now.