By the end of this post, I hope you’ll take existential risk more seriously. I hope you will feel as viscerally as I do the stakes of human survival and flourishing.
When I tell people, “I think AI is an existential risk”, usually nothing changes about their expression. They nod along, same blank face, thinking about what they’re going to have for lunch today. I want to take them by the shoulders and shake them and ask, “Are you listening? Do you know what I mean when I say ‘existential risk’? Why do you accept what I say so passively?”
When I say existential risk, I’m talking about outcomes that curtail humanity’s development forever — human extinction, permanent totalitarianism, or some crippling catastrophe that we can’t rebuild civilization up from. Do you understand, in your heart, the stakes of this?
I think when I talk about existential risk, particularly human extinction, it doesn’t feel real to people. The words “existential risk” glide in one ear and out the other — an interesting idea to consider, but abstract compared to the pressing issues today.
Let me say: existential risk is real and possible. I’m going to focus on extinction as an outcome that’s most salient to me.
99% of species that have ever lived have gone extinct. Humanity, too, can just go extinct. We can all die. This is possible! There is no force of the universe, by my eyes, that protects us. There is no rescuer, no mother or father, who will magically swoop in to save us in our time of need. It is possible for us to exist — and then one day, not. The world was barren of meaning before we existed; the world can be barren once again.
Let me clarify as well that human extinction is bad. It’s easy to hear human extinction and feel ambivalent. You imagine a peaceful end, a blink and then nothingness. What’s so special about humans, after all? Earth rotates without us — oh well.
But let’s be for real. You hear that an asteroid is hurtling toward Earth right now, and that it has a 10% chance of killing us all. Seriously, imagine that for 5 seconds. Your mom calls you to tell you the news. There’s a tremor in her voice. Word spreads quickly — the air is solemn on campus, or at work. National leaders congregate to figure out what to do. The best scientists across the globe are working day and night, but they’re uncertain if it’s enough. You look around for ways you can help — can you do research? can you make the scientists’ lives easier? can you be a source of stability for your loved ones? Some of your friends have decided that they can’t do anything, that they’re better off just enjoying their life and ignoring the problem. You don’t understand them. In the end, they’ll wish they did more.
Humanity is a flame, the only light in this world. We may be unique in our capacity for moral reasoning, our ability to fight against the “‘is-ness’ of man’s present nature” and aspire for the “eternal ‘ought-ness’”. We have transformed the world, both for worse and for better. But if we die, there is no chance for things to ultimately improve, to supersede the neutrality of nature. If we die, who knows how long it will take for there to exist another species that can take our place, that can feel joy and sorrow and love like us? The flame is precious, and only we can protect it.
I don't disagree with any of this, but I don't see anything particularly new or framed differently from everywhere that's talking about it.
Have you tried this pitch on those who nod along and think about lunch? That's not me, nor most of LW, so I can't tell if this would be effective or useful for them.
Have you compared the reaction you're getting (and looking for) against talking about other existential risks (say, nuclear weapons or bio-research)? It might be instructive to see if the people who are ignoring you are just ignoring ALL risks, or if they're especially uncaring about AI risk.
For context, I run a university group about AI safety, hence many ambivalent reactions in passing. I don't think this is a new framing, and agreed that most of LW is not the target audience! I was just encouraged to post this on LW from my personal blog by a friend.
By the end of this post, I hope you’ll take existential risk more seriously. I hope you will feel as viscerally as I do the stakes of human survival and flourishing.
When I tell people, “I think AI is an existential risk”, usually nothing changes about their expression. They nod along, same blank face, thinking about what they’re going to have for lunch today. I want to take them by the shoulders and shake them and ask, “Are you listening? Do you know what I mean when I say ‘existential risk’? Why do you accept what I say so passively?”
When I say existential risk, I’m talking about outcomes that curtail humanity’s development forever — human extinction, permanent totalitarianism, or some crippling catastrophe that we can’t rebuild civilization up from. Do you understand, in your heart, the stakes of this?
I think when I talk about existential risk, particularly human extinction, it doesn’t feel real to people. The words “existential risk” glide in one ear and out the other — an interesting idea to consider, but abstract compared to the pressing issues today.
Let me say: existential risk is real and possible. I’m going to focus on extinction as an outcome that’s most salient to me.
99% of species that have ever lived have gone extinct. Humanity, too, can just go extinct. We can all die. This is possible! There is no force of the universe, by my eyes, that protects us. There is no rescuer, no mother or father, who will magically swoop in to save us in our time of need. It is possible for us to exist — and then one day, not. The world was barren of meaning before we existed; the world can be barren once again.
Let me clarify as well that human extinction is bad. It’s easy to hear human extinction and feel ambivalent. You imagine a peaceful end, a blink and then nothingness. What’s so special about humans, after all? Earth rotates without us — oh well.
But let’s be for real. You hear that an asteroid is hurtling toward Earth right now, and that it has a 10% chance of killing us all. Seriously, imagine that for 5 seconds. Your mom calls you to tell you the news. There’s a tremor in her voice. Word spreads quickly — the air is solemn on campus, or at work. National leaders congregate to figure out what to do. The best scientists across the globe are working day and night, but they’re uncertain if it’s enough. You look around for ways you can help — can you do research? can you make the scientists’ lives easier? can you be a source of stability for your loved ones? Some of your friends have decided that they can’t do anything, that they’re better off just enjoying their life and ignoring the problem. You don’t understand them. In the end, they’ll wish they did more.
Humanity is a flame, the only light in this world. We may be unique in our capacity for moral reasoning, our ability to fight against the “‘is-ness’ of man’s present nature” and aspire for the “eternal ‘ought-ness’”. We have transformed the world, both for worse and for better. But if we die, there is no chance for things to ultimately improve, to supersede the neutrality of nature. If we die, who knows how long it will take for there to exist another species that can take our place, that can feel joy and sorrow and love like us? The flame is precious, and only we can protect it.