I’m personally skeptical that this is really the main thing going on, since it seems unlikely that many people are really mostly concerned for their own non-death out of selfless regard for the feelings of others. I’m also skeptical that this would be their view on a bunch more consideration.
Since I was a child I've always been convinced that life is more negatives than positives. What has always stopped me from offing myself was my unwillingness to hurt my parents and the people I love. I was afraid of death and of committing the act too of course. I'd say that at least a person out of ten that I talked to about this stuff in real life feels similarly.
An instantaneous extinction as described, for instance, in AI 2027 would be a perfect solution to my fears. I suspect that most of those who "don't really care if everyone dies" do indeed believe that.
I still see life as a net negative, but now I wish to live until we develop AGI/ASI and I hope that happens soon. That's likely to drastically change the way I experience life. I'd prefer more utopical outcomes of course, but an instantaneous extinction is definitely better than dying in several decades after a long kind of life like the one I've experienced so far. A quick-ish but not immediate extinction would be preferable too, as long as it doesn't come with too much extra sufferance.
More likely I think, AI catastrophe looks like being with your family and friends as usual, and over an extended period of time watching one another succumb to things like starvation due to having no money, illness due to having no money, or literally overheating or being crushed in the midst of a breathtakingly fast technological buildout that doesn’t need any of the specific conditions we depend on to live, doesn’t care about us, and does need to do vasts amount of computation quickly. Meanwhile, there won’t be a moment when you know this has happened—there will be increasing confusion and misinformation and decreasing resources put into making it legible to you. It will probably look more like the atrocities we are used to than the perfect, clean instantaneous end to the human story.
Why do you think this scenario is more likely?
A slow extinction like you describe it is likely to lead to a lot of unnecessary destruction/entropy: wars, riots, people bombing data centers and burning natural resources.
I don't have the presumption to understand what an hypothetical ASI might decide: your scenario is of course possible. But I suspect that for an ASI with technology indistinguishable from magic it'd be cheaper (in terms of resources and entropy) to kill everyone quickly before we have time to react, rather than letting us go wild for months or years.
A notable fraction of people respond to hearing about existential risk from AI by saying they don’t really care if everyone dies. I think the idea is often along the lines of ‘well if we are all dead, then there’s nobody to be unhappy about it’.
I’m personally skeptical that this is really the main thing going on, since it seems unlikely that many people are really mostly concerned for their own non-death out of selfless regard for the feelings of others. I’m also skeptical that this would be their view on a bunch more consideration.
So to help with the consideration—
My guess is that an important thing going on here is that the ‘everyone dying at once’ image seems kind of like a thought experiment—abstract, hypothetical, neat, not very sinister. Also, you literally can never see it, so it feels pretty surreal.
But it is interesting that we even have this assumption that everyone will die together.
It’s true that in some prominent AI catastrophe stories, a single AI system suddenly emerges fantastically more powerful than anyone else and builds technology to quickly kill everyone, perhaps before they notice.
But this doesn’t seem like the bulk of probability.
More likely I think, AI catastrophe looks like being with your family and friends as usual, and over an extended period of time watching one another succumb to things like starvation due to having no money, illness due to having no money, or literally overheating or being crushed in the midst of a breathtakingly fast technological buildout that doesn’t need any of the specific conditions we depend on to live, doesn’t care about us, and does need to do vasts amount of computation quickly. Meanwhile, there won’t be a moment when you know this has happened—there will be increasing confusion and misinformation and decreasing resources put into making it legible to you. It will probably look more like the atrocities we are used to than the perfect, clean instantaneous end to the human story.