Summary It has been argued that, if two very similar agents follow decision theories allowing for superrationality (e.g., EDT and FDT), they would cooperate in a prisoner’s dilemma (PD) (see e.g., Oesterheld 2017). But how similar do they need to be exactly? In what way? This post is an attempt at addressing these questions. This is, I believe, particularly relevant to the work of the Center on Long-Term Risk on acausal reasoning and the foundations of rational agency (see section 7 of their research agenda). I’d be very interested in critics/comments/feedback. This is the main reason why I’m posting this here. :) Normal PD Consider this traditional PD between two agents: Alice/BobCDC3, 30, 5D5, 01, 1 We can compute the expected payoffs of Alice and Bob (UAliceand UBob) as a function of p (the probability that Alice plays C) and q (the probability that Bob plays C): UAlice(p,q)=4q−pq−p+1 UBob(p,q)=4p−qp−q+1 Now, Alice wants to find p∗ (the optimal p, i.e., the p that will maximize her payoff). Symmetrically, Bob wants to find q∗. They do some quick math and find that p∗=q∗=0=0, i.e., they should both play D. This is the unique Nash equilibrium of this game. Perfect-copy PD Now, say Alice and Bob are perfect copies. How does it change the game presented above? We still have the same payoffs: UAlice(p,q)=4q−pq−p+1 UBob(p,q)=4p−qp−q+1 However, this time, p=q. Whatever one does, that’s evidence that the other does the exact same. They are decision-entangled[1]. What does that mean for the payoff functions of Alice and Bob? Well, decision theorists disagree. Let’s see what the two most popular decision theories (CDT and EDT) say, according to my (naive?) understanding: * EDT: “Alice should substitute q for p and her formula. Symmetrically, Bob should do the exact opposite in his”. * UAlice(p,q)=4p−pp−p+1 * UBob(p,q)=4q−qq−q+1 * CDT: “Alice should hold q fixed. Same for Bob and p. They should behave as if they could change their action unilateral