We cared about the Nazis not getting nuclear weapons before us.
I am sure if after WW2 we agreed with the Soviets that we would pause nuclear research and not research the hydrogen bomb, both sides would have signed the treaty and continued research covertly while hoping the other side sticks with the treaty. I don't think you need game theory to figure out that neither side could take the risk of not researching.
It seems incredibly naive to believe this exact process would not also play out with AI.
We cared about the Nazis not getting nuclear weapons before us.
I am sure if after WW2 we agreed with the Soviets that we would pause nuclear research and not research the hydrogen bomb, both sides would have signed the treaty and continued research covertly while hoping the other side sticks with the treaty. I don't think you need game theory to figure out that neither side could take the risk of not researching.
It seems incredibly naive to believe this exact process would not also play out with AI.