The Prisoner's Dilemma is a well-studied game in game theory. Abstractly, two agents must each decide whether or not to "defect" against the other, increasing their individual score by decreasing the other's score by a larger amount.
The only Nash Equilibrium of the Prisoner's Dilemma is both players defecting, even though each would prefer the cooperate/cooperate outcome.
A popular variant is the Iterated Prisoner's Dilemma, where two agents play the Prisoner's Dilemma against each other a number of times in a row.