Many people take it for granted that government won’t do anything to address societal scale AI risk unless or until there is a catastrophic “warning shot,” where an AI goes rogue and causes some serious damage. Something like Chernobyl or 9/11, where a bunch of people die.
Many people have told me they hope for such a warning shot. This is grim.
Fortunately, I don’t think we need a warning shot. Why not? Well, here are a few reasons:
My personal experience over the past >15 years is that over time, more and more people become more and more concerned about the problems. This might not happen fast enough, but it’s been very fast since the start of 2026.
Job loss or other societal effects of AI could create political will to stop AI, even absent any loss-of-control type catastrophe.
It seems like the problem is not the people don’t care, it’s that they aren’t paying attention and/or don’t understand the situation. So things that draw attention to the issue, including less harmful warning shots like the Hugging Face Incident, but also deliberate efforts like the Statement on AI Risk or Pacing the Frontier can move the needle and, ultimately, be sufficient.
People regularly get all up in arms about things that don’t matter as much or aren’t even real issues.
But I’m writing now to put forth a new theory: The examples people are comparing AI risk with are different. They are using a bad “reference class”.
The main difference is: A single AI incident could kill everyone, and it is far from guaranteed that we would get such a warning shot in advance. If people — including politicians — realize this, they will want to act to prevent this.
With the other examples people point at, politicians mostly could afford to wait until there was a warning-shot disaster. Their lives weren’t clearly on the line. They may even have preferred not to act, and just had their hand forced by the public outcry.
But AI risks threaten politicians and those with power. If they realize this, we should expect them to view it like a possible war with a rival power: extremely seriously.
Perhaps the best comparison in this regard is nuclear weapons, where governments seem to have quickly realized their destructive potential, before seeing a single detonation. After WWII, the US and the USSR both agreed (at least ostensibly) that they wanted complete nuclear disarmament, they just didn’t agree on the details.
Or, there’s the example from fiction: an alien invasion.
If you think that people need to see a catastrophe to understand that AI is dangerous, you must not think much of their intelligence. It’s true that a lot of people aren’t paying much attention, or are being misled by false narratives. But those are fixable problems, and we don’t actually need a bunch of people to die to fix them.
If you think we do, imagine explaining to people: “We knew you were at risk, but we just figured you were too stupid to understand that unless we let a bunch of you die.”
Thanks for reading The Real AI! Subscribe for free to receive new posts and support my work.
Many people take it for granted that government won’t do anything to address societal scale AI risk unless or until there is a catastrophic “warning shot,” where an AI goes rogue and causes some serious damage. Something like Chernobyl or 9/11, where a bunch of people die.
Many people have told me they hope for such a warning shot. This is grim.
Fortunately, I don’t think we need a warning shot. Why not? Well, here are a few reasons:
My personal experience over the past >15 years is that over time, more and more people become more and more concerned about the problems. This might not happen fast enough, but it’s been very fast since the start of 2026.
Job loss or other societal effects of AI could create political will to stop AI, even absent any loss-of-control type catastrophe.
It seems like the problem is not the people don’t care, it’s that they aren’t paying attention and/or don’t understand the situation. So things that draw attention to the issue, including less harmful warning shots like the Hugging Face Incident, but also deliberate efforts like the Statement on AI Risk or Pacing the Frontier can move the needle and, ultimately, be sufficient.
People regularly get all up in arms about things that don’t matter as much or aren’t even real issues.
But I’m writing now to put forth a new theory: The examples people are comparing AI risk with are different. They are using a bad “reference class”.
The main difference is: A single AI incident could kill everyone, and it is far from guaranteed that we would get such a warning shot in advance. If people — including politicians — realize this, they will want to act to prevent this.
With the other examples people point at, politicians mostly could afford to wait until there was a warning-shot disaster. Their lives weren’t clearly on the line. They may even have preferred not to act, and just had their hand forced by the public outcry.
But AI risks threaten politicians and those with power. If they realize this, we should expect them to view it like a possible war with a rival power: extremely seriously.
Perhaps the best comparison in this regard is nuclear weapons, where governments seem to have quickly realized their destructive potential, before seeing a single detonation. After WWII, the US and the USSR both agreed (at least ostensibly) that they wanted complete nuclear disarmament, they just didn’t agree on the details.
Or, there’s the example from fiction: an alien invasion.
If you think that people need to see a catastrophe to understand that AI is dangerous, you must not think much of their intelligence. It’s true that a lot of people aren’t paying much attention, or are being misled by false narratives. But those are fixable problems, and we don’t actually need a bunch of people to die to fix them.
If you think we do, imagine explaining to people: “We knew you were at risk, but we just figured you were too stupid to understand that unless we let a bunch of you die.”
Thanks for reading The Real AI! Subscribe for free to receive new posts and support my work.