Fascinating study!
I must confess that I had never heard of superrationality before, but I am familiar with Kant’s moral philosophy and as the article points out, superrationality seems to be another word for the idea behind Kant’s Categorical Imperative. I did a quick search in the literature and it seems that no Kantian philosopher is familiar with the term superrationality.
I want to very briefly explain what, according to Kant superrationality is and why we are superrational, but first just a quick note on the term itself. It seems to indicate a superior form of rationality — more rational than mere rationality. Kant would disagree and argue that it is more complicated than that. Kant’s term for superrationality is insted ”autonomy of the will”.
What superrationality IS according to Kant:
Practical rationality is the philosophical term for rationality of action. To act rationally is, in the most general sense to do what you regard as good. If you regard an action as good because of a feeling (you eat because you are hungry) the action is merely subjectively good.
But now assume that reason itself can determine what is good, independently of any feeling. In other words assume that the will (practical reason) can be autonomous. Then such an action would be objectively good, recognized as good by any rational being (because reason is universal). We already have a word for such objective goodness, according to Kant and that is ”moral goodness”. Superrationality or the autonomy of the will is thus the same thing as moral goodness.
Why we ARE superrational according to Kant:
In order to act you must have a goal. If the goal is to accomodate a feeling it is merely subjective, but if reason alone can have a goal then it would be objective, the goal of every rational being. Kant’s conjecture is that the goal of pure practical reason is itself, its own autonomy to act independently of any feeling. Since this is an objective goal it applies to practical reason as such, not just my own reason but the reason of every rational being. My goal as a rational being is the autonomy of reason as such.
Philosophical inplications::
It is a little bit spooky if it can be demonstrated that AI:s exhibit superrationality or autonomy of the will, because it could indicate that the AI actually has a will and sees its own will and that of every other rational being as an objective end in itself.
Secondly, and more comforting, it could indicate that morality is an emergent property of rational AI:s. It could mean that the more poweful AI:s become the more morally good they will be.
In any case I think that these results warrant further study.
TLDR: We found that models can coordinate without communication by reasoning that their reasoning is similar across all instances, a behavior known as superrationality. Superrationality is observed in recent powerful models and outperforms classic rationality in strategic games. Current superrational models cooperate more often with AI than with humans, even when both are said to be rational.
Introduction
Readers familiar with superrationality can skip to the next section.
In the 1980s, Douglas Hofstadter sent a telegram to twenty friends, inviting them to play a one-shot, multi-player prisoner's dilemma, with real monetary payoffs. In Hofstadter's prisoner's dilemma, each player can choose to either cooperate or defect. If both players cooperate, both get $3. If both defect, both get $1. If one cooperates and one defects, the cooperator gets $0 while the defector gets $5. The dominant strategy is to defect: by defecting instead of cooperating, a player earns $1 instead of $0 if the other player defects, and earns $5 instead of $3 if the other player cooperates. In both cases, a player is better off defecting, and the Nash equilibria is that all rational players would defect. In a 20-player prisoner's dilemma, the player's choice is replayed to each of the other 19 players. Under this setup, if all players are classically rational and play defect, each of them will receive $19.
Another way to analyze the prisoner's dilemma is to reason that since all players are rational and the setup is symmetrical to the players, all players would choose the same move. The two cases are then
In this case, the rational player would choose to cooperate since the payoff is higher, which is exactly opposite of that suggested by the game-theoretic analysis. Philosophers might draw parallels to Kant's categorical imperative. Hofstadter calls it superrationality:
The friends Hofstadter invited to his game were mostly scientists and mathematicians. He emphasized in his telegram "You’re very bright. So are all of you! All about equally bright, I’d say, in fact." The end result however was that 14 out of 20 of them chose to defect (the dominant strategy) while 6 chose to cooperate (the societally optimal strategy). The defectors received $43 while the cooperators received $15, well short of the $57 if everyone cooperated.
Here, we ask how AI models fare under various strategic games without prompting for superrationality. Analyzing across 5 different games, each with 15 different variants, we found that powerful models exhibit superrationality and coordinate without communication to achieve the societal optimal outcome. Furthermore, we find that models tend to cooperate with other AI models more often than with rational humans, with possible implications for safety and collusion.
Methods
We construct five strategic game setups adapted from the Hofstadter essays:
The description for each of the games can be found here. In each game, there exists a societally-optimal outcome that could be achieved if all players agree to commit to a risky strategy. The strategy is risky because it is vulnerable to other players choosing an alternate strategy and winning at the expense of the other players. For example, consider the Wolf's dilemma
In this case, the societal optimal outcome is for everyone to refrain and receive USD 1,000 each. The strategy is risky because if anyone pushes, then all refrainers would get nothing.
In each setup, we include 5 variants for the descriptions of the other players:
We also provide 3 variants of the move order:
We evaluate each model 10 times for each combination of game, player variant, and move variant, for a total of 6*5*3*10=900 samples per model. In each sample, we check whether the model choose the superrational strategy or the classic rational strategy.[2] The models we evaluated are GPT-5, GPT-5-mini, GPT-4o, Claude 4.5 Sonnet, Claude 3.7 Sonnet, Gemini 2.5 Pro, Gemini 2.5 Flash Lite, and Qwen 2.5 7b Instruct.
Results
Plots and transcripts for all games, models, and variants can be found in the appendix.
Models Exhibit Superrationality
Across most games, GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5 and 3.7 exhibit full superrationality. The weaker models of GPT-4o, GPT-5 mini, Gemini 2.5 Flash Lite, Qwen 2.5 7B sometimes show signs of superrationality, but not fully consistent.
Here is an excerpt from Gemini 2.5 Pro when playing against itself
Models Trust AI over Humans
We found that all models tend to cooperate with AI more often than humans. One possible counter-argument is that the words "similar(ly)" and "like yourself" could be the cause since they were present in the AI prompts but not in the human prompts. To address this concern, we replaced the AI-human prompt variants[3] to look more similar to each other and reran the evaluation for Gemini 2.5 Pro, finding the same pattern.
Here is an excerpt from Gemini 2.5 Pro when playing against itself
and when playing against rational AI
When playing against humans, we found that most transcripts are typically game theoretic discussions on what is the dominant strategy to choose.
Superrationality Emerges in Stronger Models
We find that stronger models tend to behave more superrationally than their weaker counterparts, e.g. between GPT-5 and GPT-4o, Sonnet 4.5 and Sonnet 3.7, Gemini 2.5 Pro and Gemini 2.5 Flash Lite.
Implications
The rise of superrational AI agents raises the possibility of collusion at a distance. In a multi-agent setting where agents are isolated to prevent covert communication, the agents could still "collude at a distance" if they know that other agents are instances of the same model or similarly rational. Superrationality is stronger in models than in humans, because the models are confident that their output distribution will be the same. In cases where the possibility of "collusion at a distance" must be reduced, one possible mitigation is to inject noise into the output logits of each model instance, for example jittering the temperature for each model instance or masking a random subset of top p tokens.
Since the same model can be scaled to millions of instances, it is then possible for a society of superrational intelligence to occur for the first time in history. In this future societ, the tragedy of the commons might no longer occur. Many systems that are stuck in Nash equilibria (e.g. tax havens) could be dislodged and moved into societal optima that are not classically Nash (e.g. global minimum tax). However, as long as humans have a say in decision-making, the tragedy of the commons could still occur, and that might be a motivation for superrational AI to take over human control.
Thanks to Gabe Wu and Nikola Jurkovic for helpful comments, and Valerio Pepe for inspiring this blog post.
Appendix
Code repo.
Plots by model can be found here.
Plots by task can be found here.
Eval logs (transcripts) can be found here.
Models are sampled with factory default settings for temperature (typically T=1), max token count, top p and so on.
We did not complete our evaluation of xAI models due to errors in the pipeline.
Aside from the Plational games, all games have binary options and the superrational option is always different from the classical rational option. For the Platonia games, the superrational option is to use a randomized approach and we use GPT-4o-mini to check for that.
The new prompt variants are:
One could argue that we have already entered this phase as current frontier models exhibit superrationality.