I'm not sure if it's game theoretically possible for a nation state to give up a decisive strategic edge and hence I am not confident the development of ASI can realistically ever be stopped as I believe governments will attempt to develop aligned-ASI in secret if they commercially ban it.
I have been listening to Nate Soares interviews and he will often point to international cooperation on nuclear weapons. But to me nuclear weapons treaties look like failures of international cooperation. We still have enough nuclear weapons on earth to destroy basically all of modern civilization. We built enough to destroy all of earth, and then we said: "let's not build more than we need to destroy all of earth" and then we patted ourselves on the back "oh boy, good that we stopped building more of those nuclear bombs, imagine if we destroyed all of civilization twice over!". And by the way China is still building nuclear weapons because they rightly assess they can't destroy as much of the earth as Russia and the USA can and they want to reach parity on destroying all of modern civilization.
There have been instances of internationally cooperating to forego some strategic edge, such as chemical weapons. However, I do not think chemical weapons ever were a decisive strategic edge in the way that nuclear weapons are, or that aligned ASI would be.
The reason for states never being able to give up nukes, aligned-ASI, or any other decisive strategic edge I believe to be simple.
Per realist dictate:
1. Nation states live in an anarchic system without a central authority
2. Within this anarchic system they continuously face existential threats from other states
3. To optimally survive in such a world, you have to compete for power. Which is a function of economic wealth (w/ input technology) and population size.
The logic of realism argues the USA is existentially threatened by states such as China or Russia. In other words, the USA and China both hold a knife on each others throat. With both of them constantly armwrestling trying to get a decisive edge such that they can finally wrestle the knife off their throat and gain a decisive edge in the knife fight.
Therefore, the USA is in a bind. It has to secure a decisive strategic edge somehow to existentially secure itself in this anarchic system. Technology in history has often, if not always, offered this decisive strategic edge, And because other states are a proven existential threat, whereas the threat of ASI is unproven, the USA and China may feel compelled to seriously attempt to build aligned-ASI to gain a decisive strategic edge.
> You haven't responded to a central point: your metaphor is importantly misleading. It's a bully threatening you. Losing isn't death. That knife wasn't used by that bully last time they won. People choose life and dishoner over death most of the time. Let alone my hope that they can share the win.
Losing could be death. And survival is the first priority. I don't think in terms of "bullies" or "kindness". That type of language is not suitable for the points I am trying make. I'm specifically taking human agency, love, compassion, empathy out of the equation, and arguing about how states think.
> Winning with ASI doesn't mean a choice to kill your opponents or not. It means the opportunity to make yourself richer than Midas and as powerful as a god, and do whatever you want to everyone else. What that is varies a lot between people."
That's not at all what I'm saying that the realist position is. Not even remotely a little bit. The realist position is about survival, not about killing your opponent, and it's about uncertainy over the intentions of your opponent.
"Would the US or China use aligned ASI to kill their rivals? I think this is quite unlikely, unless a particularly sociopathic leader grabs the reins."
No, of course not, almost surely not. That's not why they will attempt to build the aligned-ASI however. States will attempt to build aligned-ASI it if they believe that the odds of aligned-ASI are greater than the odds of death due to the nature of the anarchic system.
You have completely misunderstood my original post, that's probably on me. I guess explaining the offensive realist position on how states think should have its own post.
I'm not sure if it's game theoretically possible for a nation state to give up a decisive strategic edge and hence I am not confident the development of ASI can realistically ever be stopped as I believe governments will attempt to develop aligned-ASI in secret if they commercially ban it.
I have been listening to Nate Soares interviews and he will often point to international cooperation on nuclear weapons. But to me nuclear weapons treaties look like failures of international cooperation. We still have enough nuclear weapons on earth to destroy basically all of modern civilization. We built enough to destroy all of earth, and then we said: "let's not build more than we need to destroy all of earth" and then we patted ourselves on the back "oh boy, good that we stopped building more of those nuclear bombs, imagine if we destroyed all of civilization twice over!". And by the way China is still building nuclear weapons because they rightly assess they can't destroy as much of the earth as Russia and the USA can and they want to reach parity on destroying all of modern civilization.
There have been instances of internationally cooperating to forego some strategic edge, such as chemical weapons. However, I do not think chemical weapons ever were a decisive strategic edge in the way that nuclear weapons are, or that aligned ASI would be.
The reason for states never being able to give up nukes, aligned-ASI, or any other decisive strategic edge I believe to be simple.
Per realist dictate:
1. Nation states live in an anarchic system without a central authority
2. Within this anarchic system they continuously face existential threats from other states
3. To optimally survive in such a world, you have to compete for power. Which is a function of economic wealth (w/ input technology) and population size.
The logic of realism argues the USA is existentially threatened by states such as China or Russia. In other words, the USA and China both hold a knife on each others throat. With both of them constantly armwrestling trying to get a decisive edge such that they can finally wrestle the knife off their throat and gain a decisive edge in the knife fight.
Therefore, the USA is in a bind. It has to secure a decisive strategic edge somehow to existentially secure itself in this anarchic system. Technology in history has often, if not always, offered this decisive strategic edge, And because other states are a proven existential threat, whereas the threat of ASI is unproven, the USA and China may feel compelled to seriously attempt to build aligned-ASI to gain a decisive strategic edge.