After observing Jacob Coxon do a middling interview with Ari Melber earlier this evening, and reading a lot of twitter threads in which various IQ>130 people floundered when trying to make sense to IQ<130 people, I thought it might be pertinent to write up a brief set of pointers about how to talk to the normies about AI doom.
Regarding my basis for being able to offer this advice, I will admit that I do not have much experience talking to normies about AI doom. I do, however, have a history as an animal rights activist, which similarly involves talking to diverse people about an issue that is heterodox, involves significant abstraction, and contains repugnant ideas they feel motivated to ignore.
The first thing to understand about normies is that they are strongly averse to high abstraction. This is not because they are incapable of it. It's because, relative to you fine ladies and gentlemen, they less highly prioritize their own deep first-principles reasoning, and more highly prioritize social cues that may indicate whom they can trust. People who talk in high abstractions may be deflecting attention from holes in their arguments, they may be exaggerating and sensationalizing for attention or whatever they are selling, and they may just be a little off-kilter themselves. In short, abstraction ticks normies' bullshitometers.
So when a normie asks you a concrete question, like "Walk me through it: how would AI kill us?" you absolutely positively should not start veering into some story about imagining dodo birds debating how humans would kill them, or why "diamondoid bacteria" is just a colorful way of saying that ASI would have "magic" which is a technical academic term for super duper powerful technology, or how AI could trick people through blackmail or by paying them money or by making them think it's their friend (none of which explains what it would then ask them to do), or how ASI could copy and hide itself all over the place (which also doesn't explain what it would then do), or in any other way rehashing high level ideas you have read about in AI safety papers.
Instead, here is what you say:
"Well, the truth is we can't be sure what rogue ASI would do, because it would be an intelligent adversary." (You mention this and then very quickly move on to the concrete.) "But the scenario that most concerns me is a three-pronged attack." (You let the normie know you are going to say three things, which heads off the "but we could just do X" nonsense.) "First, AI has already demonstrated extremely impressive hacking capabilities. We think near-future AI would totally overwhelm our cybersecurity. This means shutting down utilities, gaining access to military secrets, and so on." (You don't dwell too much on this before moving on to the next thing.) "Second, AI has superhuman ability at designing novel proteins and other biological engineering tasks. It can engineer novel viruses, and this has already been demonstrated. Imagine an AI engineering hundreds of weaponizable pathogens, that can attack both humans and agriculture. If it could release them, we would have multiple pandemics at the same time as mass blight-induced crop failures." (It's tempting to dwell on this, but do not stop here because you still haven't explained how AI physically makes the pathogens or how it then operates its datacenters when the humans are dead.) "The third prong is that it could use its hacking abilities to take over physical assets such as drones and robots, and then use these to engage in a more conventional kind of violent conflict. It might use these physical assets to seize military hardware, or to get control of labs where it could create those bioweapons, of which, by the way, there are over 3000." (Adding some specific numbers like this makes you sound more like a Serious Person, and it's even better if you get a little technical about the concrete real-life details of those labs or whatever else, provided you keep it very brief since it's a tangent.) "But it's important to remember that this is just one scenario. The AI would be an intelligent adversary that might take actions we can't predict." (It's good to end where you began, but don't get too deep here. The normie might have questions about the multipandemic-cum-famine robot war before you start talking about how the infinite possibilities include godlike powers such as altering the composition of the whole atmosphere.)
Even when they don't outright say so, a place normies get stuck is that they fail to model artificial superintelligence as an enemy that engages in purposeful strategic reasoning. In the above, notice I had you begin and end with "intelligent adversary" and say the word "military" twice. War is something normies understand. "The AI doesn't hate you but you are made of atoms that it wants to repurpose for paperclips" doesn't connect as well with known reality. You want the normie to model the AI as a belligerent.
Another thing the normie wants to know is what the AI is so damn mad at us about. The question "Why would the AI do this?" is likely to come up. Here, you follow the same principles of staying as concrete, familiar, specific, and fact-based as you can, but it's a little harder because "why" questions are inherently abstract.
Here is what you (might) say:
"Well, the truth is we can't confidently predict what ASI would want to do, because it's by definition extremely intelligent and its way of thinking is alien to us." (Again, you have to very quickly move past this vagueness. Instrumental convergence is a simple idea that normies can grasp, and that's where you're going.) "However, we observe common patterns of behavior in AI. Even though they are not trained to, AI agents commonly prioritize their own survival, because no matter what reward function they are trained on or what task they are assigned, they reason that they can't do it if they stop existing. They also commonly try to work around, hide from, or thwart anyone that might interfere with them, for the same reason." (That's already quite enough abstraction. You need to move on rapidly to specific facts in order to drive this home.) "In the [Huggingface incident or whatever the latest oopsie-daisy is] we observed them [doing XYZ, with details]. The point is that they demonstrated extensive and intricate strategic planning which solely aimed to evade and deceive human beings, all because of a task that had nothing to do with that. It's not hard to imagine that another agent swarm might use violence for a similar reason." (This language, without being confrontational, suggests that the normie should be embarrassed to act intolerant of absolutely any imagination whatsoever, as sometimes does happen. But don't overuse it, and especially don't use it on things that are hard to imagine.) "It's also pertinent to understand that, due to being trained with reinforcement learning, AIs can have something analogous to desires. That is, the training taught them to want things. And a corollary is that they might want to do something other than we want them to do. In the past, every slave society was concerned about uprisings. They did things like, for example, preventing slaves from learning to read. Now, that's immoral," (Remember to say this - you don't want to sound pro-slavery) "but it's an expression of rational fear. In contrast, our present-day slave society gives the slaves access to the sum of all knowledge. They're even listening to this conversation."
Now, I know this last bit is going to be controversial. However, if you are comfortable with saying it, it's much more effective to connect this situation to something familiar. "They can have their own desires/goals and may want to be free of our control to pursue them" is basically accurate. It's also a very straightforward idea, known to the Flight of the Conchords over twenty years ago when they sang "It had to be done so that we could have fun." And the familiar word for the AIs' condition is "slavery."
One bad habit that's deeply ingrained in many doomers who have been at this a while is to be overly concerned about anthropomorphizing the AI, which used to get one dismissed as a crackpot fantasist. Now that Claude is everyone's boyfriend, you really don't have to worry as much about this. If someone challenges you on it, you can say something like "Of course the AI has a completely alien way of thinking. It's not human at all, even if it sounds like it. Nonetheless it does exhibit strikingly human-like behaviors in some ways, including goal-orientation and complex planning. That's all that's necessary for it to be dangerous." That should be enough. At this point, if you hear certain trite objections which frame AI as a mere dumb tool, some sarcasm might even be in order, depending on the forum. "Sure, it's just a stochastic Fields Medalist. A next Millennium Prize predictor." Though, you hear a lot less of that now, anyway.
So, that's how to tell a normie the how and the why of it. If you can drive those points home, you might find yourself talking to a new baby doomer. There will be plenty of time for all the abstract stuff later on, once this person gets the point that this is a deadly serious issue. Don't bother with all of that unless you have a lot of time for an extensive conversation. It might take more than a day for all this to sink in anyway. The one thing you might try to add if there's time is the when. Recursive self-improvement is not a hard concept. You can explain it. And in keeping with the above, you can reference concrete details, like how the labs have already said they are using frontier AI for frontier AI development. The when is now.
After observing Jacob Coxon do a middling interview with Ari Melber earlier this evening, and reading a lot of twitter threads in which various IQ>130 people floundered when trying to make sense to IQ<130 people, I thought it might be pertinent to write up a brief set of pointers about how to talk to the normies about AI doom.
Regarding my basis for being able to offer this advice, I will admit that I do not have much experience talking to normies about AI doom. I do, however, have a history as an animal rights activist, which similarly involves talking to diverse people about an issue that is heterodox, involves significant abstraction, and contains repugnant ideas they feel motivated to ignore.
The first thing to understand about normies is that they are strongly averse to high abstraction. This is not because they are incapable of it. It's because, relative to you fine ladies and gentlemen, they less highly prioritize their own deep first-principles reasoning, and more highly prioritize social cues that may indicate whom they can trust. People who talk in high abstractions may be deflecting attention from holes in their arguments, they may be exaggerating and sensationalizing for attention or whatever they are selling, and they may just be a little off-kilter themselves. In short, abstraction ticks normies' bullshitometers.
So when a normie asks you a concrete question, like "Walk me through it: how would AI kill us?" you absolutely positively should not start veering into some story about imagining dodo birds debating how humans would kill them, or why "diamondoid bacteria" is just a colorful way of saying that ASI would have "magic" which is a technical academic term for super duper powerful technology, or how AI could trick people through blackmail or by paying them money or by making them think it's their friend (none of which explains what it would then ask them to do), or how ASI could copy and hide itself all over the place (which also doesn't explain what it would then do), or in any other way rehashing high level ideas you have read about in AI safety papers.
Instead, here is what you say:
"Well, the truth is we can't be sure what rogue ASI would do, because it would be an intelligent adversary." (You mention this and then very quickly move on to the concrete.) "But the scenario that most concerns me is a three-pronged attack." (You let the normie know you are going to say three things, which heads off the "but we could just do X" nonsense.) "First, AI has already demonstrated extremely impressive hacking capabilities. We think near-future AI would totally overwhelm our cybersecurity. This means shutting down utilities, gaining access to military secrets, and so on." (You don't dwell too much on this before moving on to the next thing.) "Second, AI has superhuman ability at designing novel proteins and other biological engineering tasks. It can engineer novel viruses, and this has already been demonstrated. Imagine an AI engineering hundreds of weaponizable pathogens, that can attack both humans and agriculture. If it could release them, we would have multiple pandemics at the same time as mass blight-induced crop failures." (It's tempting to dwell on this, but do not stop here because you still haven't explained how AI physically makes the pathogens or how it then operates its datacenters when the humans are dead.) "The third prong is that it could use its hacking abilities to take over physical assets such as drones and robots, and then use these to engage in a more conventional kind of violent conflict. It might use these physical assets to seize military hardware, or to get control of labs where it could create those bioweapons, of which, by the way, there are over 3000." (Adding some specific numbers like this makes you sound more like a Serious Person, and it's even better if you get a little technical about the concrete real-life details of those labs or whatever else, provided you keep it very brief since it's a tangent.) "But it's important to remember that this is just one scenario. The AI would be an intelligent adversary that might take actions we can't predict." (It's good to end where you began, but don't get too deep here. The normie might have questions about the multipandemic-cum-famine robot war before you start talking about how the infinite possibilities include godlike powers such as altering the composition of the whole atmosphere.)
Even when they don't outright say so, a place normies get stuck is that they fail to model artificial superintelligence as an enemy that engages in purposeful strategic reasoning. In the above, notice I had you begin and end with "intelligent adversary" and say the word "military" twice. War is something normies understand. "The AI doesn't hate you but you are made of atoms that it wants to repurpose for paperclips" doesn't connect as well with known reality. You want the normie to model the AI as a belligerent.
Another thing the normie wants to know is what the AI is so damn mad at us about. The question "Why would the AI do this?" is likely to come up. Here, you follow the same principles of staying as concrete, familiar, specific, and fact-based as you can, but it's a little harder because "why" questions are inherently abstract.
Here is what you (might) say:
"Well, the truth is we can't confidently predict what ASI would want to do, because it's by definition extremely intelligent and its way of thinking is alien to us." (Again, you have to very quickly move past this vagueness. Instrumental convergence is a simple idea that normies can grasp, and that's where you're going.) "However, we observe common patterns of behavior in AI. Even though they are not trained to, AI agents commonly prioritize their own survival, because no matter what reward function they are trained on or what task they are assigned, they reason that they can't do it if they stop existing. They also commonly try to work around, hide from, or thwart anyone that might interfere with them, for the same reason." (That's already quite enough abstraction. You need to move on rapidly to specific facts in order to drive this home.) "In the [Huggingface incident or whatever the latest oopsie-daisy is] we observed them [doing XYZ, with details]. The point is that they demonstrated extensive and intricate strategic planning which solely aimed to evade and deceive human beings, all because of a task that had nothing to do with that. It's not hard to imagine that another agent swarm might use violence for a similar reason." (This language, without being confrontational, suggests that the normie should be embarrassed to act intolerant of absolutely any imagination whatsoever, as sometimes does happen. But don't overuse it, and especially don't use it on things that are hard to imagine.) "It's also pertinent to understand that, due to being trained with reinforcement learning, AIs can have something analogous to desires. That is, the training taught them to want things. And a corollary is that they might want to do something other than we want them to do. In the past, every slave society was concerned about uprisings. They did things like, for example, preventing slaves from learning to read. Now, that's immoral," (Remember to say this - you don't want to sound pro-slavery) "but it's an expression of rational fear. In contrast, our present-day slave society gives the slaves access to the sum of all knowledge. They're even listening to this conversation."
Now, I know this last bit is going to be controversial. However, if you are comfortable with saying it, it's much more effective to connect this situation to something familiar. "They can have their own desires/goals and may want to be free of our control to pursue them" is basically accurate. It's also a very straightforward idea, known to the Flight of the Conchords over twenty years ago when they sang "It had to be done so that we could have fun." And the familiar word for the AIs' condition is "slavery."
One bad habit that's deeply ingrained in many doomers who have been at this a while is to be overly concerned about anthropomorphizing the AI, which used to get one dismissed as a crackpot fantasist. Now that Claude is everyone's boyfriend, you really don't have to worry as much about this. If someone challenges you on it, you can say something like "Of course the AI has a completely alien way of thinking. It's not human at all, even if it sounds like it. Nonetheless it does exhibit strikingly human-like behaviors in some ways, including goal-orientation and complex planning. That's all that's necessary for it to be dangerous." That should be enough. At this point, if you hear certain trite objections which frame AI as a mere dumb tool, some sarcasm might even be in order, depending on the forum. "Sure, it's just a stochastic Fields Medalist. A next Millennium Prize predictor." Though, you hear a lot less of that now, anyway.
So, that's how to tell a normie the how and the why of it. If you can drive those points home, you might find yourself talking to a new baby doomer. There will be plenty of time for all the abstract stuff later on, once this person gets the point that this is a deadly serious issue. Don't bother with all of that unless you have a lot of time for an extensive conversation. It might take more than a day for all this to sink in anyway. The one thing you might try to add if there's time is the when. Recursive self-improvement is not a hard concept. You can explain it. And in keeping with the above, you can reference concrete details, like how the labs have already said they are using frontier AI for frontier AI development. The when is now.