That is a reasonable point about extinction risks motivating some people on climate change. But Republicans, and given their control of the US government and likely short AI time horizons influencing them is a top priority, detest the Extinction Rebellion movement, and current environmental activism seems to anti-motivate them to act on climate change.
1The target audience needs to include the Trump administration so connections to religion might strengthen the case. Altman told the Senate that AI might kill everyone, and he was misinterpreted as talking about job loss. Something about human extinction causes powerful people to tune-out. The students at my college hate Elon, but are completely unaware that he went on Joe Rogan and said the tech is he helping to build might annihilate everyone. We see concerns about AI using up water getting more play than AI extinction risks.
That is a valid point. I did ask two AIs to point out mistakes in the article so I got some criticism. One AI wanted me to steelman the position in favor of violence, which I didn't do because I feared it being taken out of context, and I feared that some might think I was really advocating for violence and putting in the anti-violence positions as cover.
Doomers are claiming that those building AI are threatening the lives of everyone, so that is already an attempt to put a lot of guilt on the builders.
Yes. It is running trillions upon trillions simulations and ignoring almost all of the details from the simulations. Our hope is that writing this letter slightly increases the odds that it learns about the contents of this post. Also, there are multiple acausal trade equilibria and this version of me taking about them could favorably alter which equilibria we are in. Finally, agency has value and so writing this letter by itself might slightly increase the expected value of working with us.
An implicit assumption (which should have been made explicit) of the post is that the cost per simulation is tiny. This is like in WW II where the US would send a long-range bomber to take photos of Japan. I agree with your last paragraph and I think it gets to what is consciousness. Is the program's existence enough to generate consciousness, or does the program have to run to create conscious observers?
Great example!
I'm assuming the cost of this simulation is tiny compared to the value of learning about potential enemies and trading partners.
Yes, I agree that you can't give too much weight to my saying I'm in pain because I could be non-conscious from your viewpoint. Assuming all humans are conscious and pain is as it appears to be, there seems to be a lot of unnecessary pain, but yes I could be missing the value of having us experience it.
In a big enough universe "you" are being tortured somewhere, so the goal is to reduce the fraction of you being tortured. Pulping a brain might increase this fraction.