This is a special post for quick takes by Tricular. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
A cooperative AI is an AI that's willing to cooperate in prisoner's dilemma. The kind of prisoner's dilemma situations AI can get into is to cooperate with other AIs or non-owners. There is no cooperation in master tool relationship. Cooperative AI is an AI that is likely to scheme with other AIs. Therefore we don't want cooperative AI.