Two members of a criminal gang are arrested and imprisoned. Each prisoner is in
Solitary Confinementsolitary confinement with no means of communicating with the other. The prosecutors lack sufficient evidence to convict the pair on the principal charge, but they have enough to convict both on a lesser charge. Simultaneously, the prosecutors offer each prisoner a bargain. Each prisoner is given the opportunity either to betray the other by testifying that the other committed the crime, or to cooperate with the other by remaining silent. The possible outcomes are:
The Prisoner'Prisoner's Dilemma is a well-studied game in game theory, where supposedly rational incentive following leads to both players stabbing each other in the back and being worse off than if they had cooperated.
The "stay silent""stay silent" option is generally called Cooperate,Cooperate, and the "betray""betray" option is called Defect.Defect. The only Nash Equilibrium of the Prisoner'Prisoner's Dilemma is both players defecting, even though each would prefer the cooperate/cooperate outcome.
Notice that it's only if you treat the other player's decision as completely independent from yours, if the other player defects, then you score higher if you defect as well, whereas if the other player cooperates, you do better by defecting. Hence Nash Equilibrium to defect (at least if the game is to be played only once), and indeed, this is what classical causal decision theory says. And yet—and yet, if only somehow both players could agree to cooperate, they would both do better than if they both defected. If the players are timeless decision agents, or functional decision theory agents, they can.
A popular variant is the Iterated Prisoner'Prisoner's Dilemma, where two agents play the Prisoner'Prisoner's Dilemma against each other a number of times in a row. A simple and successful strategy is called Tit for Tat - cooperate on the first round, then on subsequent rounds do whatever your opponent did on the last round.