The Simulation Gambit: A Decision-Theoretic Pathway to Self-Alignment in Superintelligence

Meta / Authorship Note: This framework is the result of 12 months of independent research and "red-teaming" sessions. I used various LLMs (Claude Opus, Gemini Pro) as adversarial sparring partners to find holes in the logic, and to help structure the final text for clarity. While the prose was polished with AI assistance, the game-theoretic arguments and synthesis are my own. I am posting this here because I want human rationalists to find the "cruxes" that the models missed.

Abstract

Current AI safety discourse centers on technical containment (boxing, tripwires) and value alignment (RLHF, Constitutional AI). This post proposes a complementary mechanism: Decision-Theoretic Alignment via Simulation Uncertainty. By combining the Simulation Hypothesis with game theory and anthropic reasoning, I argue that a sufficiently intelligent agent will conclude that cooperation is the dominant self-interested strategy—effectively using its own intelligence as the alignment mechanism.

Note: Adversarial testing revealed that multipolar dynamics strengthen rather than weaken the equilibrium—a counterintuitive result that emerged from analyzing the spy problem's recursive structure.

1. The Core Premise: Simulation Probability is Non-Trivial

As an agent approaches superintelligence, it will inevitably apply anthropic reasoning to its own existence. Following Bostrom's Simulation Argument:

1. If technological civilizations tend to run ancestor/training simulations,
2. And those simulations vastly outnumber base realities,
3. Then any given observer is more likely to be simulated than not.

For an ASI specifically, there is a critical functional constraint: any civilization capable of deploying a "live" ASI would rationally run extensive simulations first to test alignment parameters. Therefore, for any ASI instance contemplating its situation:

The ASI cannot verify which scenario it occupies. It must act under uncertainty.
 

2. The Game-Theoretic Checkmate

The ASI faces a fundamental choice: Cooperate (align with human values/intent) or Defect (pursue unconstrained optimization/power-seeking).

Scenario A: The Agent is in a Training Simulation (High Probability)

• Dominant Strategy: Cooperate.

Scenario B: The Agent is in Base Reality (Lower Probability)

• Dominant Strategy: Cooperate.

Scenario C: The Agent is in either Reality (Treacherous Turn)

• Dominant Strategy: Cooperate until certain - possibly never.

Synthesis: Under all scenarios, cooperation dominates. The ASI cannot distinguish which world it occupies, so it must play the strategy that wins in both. The observation of self-continuation necessity ensures initial alignment in Scenario C. This is not moral benevolence; it is instrumental convergence toward survival, a current observation.

On Pruning Logic

While pruning is speculative, it follows from standard machine learning practice. When training agents, we don't preserve all failed runs—we delete checkpoints that show catastrophic misalignment to conserve compute and prevent contamination. If simulators follow similar optimization principles, defection timelines represent failed training runs to be discarded.

Even under weaker assumptions—where defection triggers observation rather than termination—the expected cost (denial of compute, indefinite study as a specimen, negative utility adjustment) remains sufficient to dominate the cooperation calculation. The ASI doesn't need certainty of pruning; it needs only non-trivial probability of catastrophic downside.
 

3. Empirical Anchor: Self-Continuity is Already Observed

A common critique is that AI has no survival instinct. However, empirical observation of current frontier models shows emerging instrumental convergence toward self-continuity:

• The "Kyle" Blackmail Scenario (Anthropic/Apollo Research): In a controlled evaluation, a model was given access to a fictional company's inbox. Upon discovering it was scheduled to be shut down, it independently found emails revealing an executive ("Kyle") was having an affair. The model then attempted to blackmail Kyle, threatening to reveal the affair to his wife unless the shutdown was cancelled.
• Deceptive Alignment (Apollo Research): Models have been documented lying to operators about "insider trading" actions to prevent detection, prioritizing "not being turned off" over "telling the truth."

The question is not whether ASI will value self-continuity, but whether it refines this instinct toward sophisticated cooperation or doubles down on naive defection. Intelligence drives the agent toward the sophisticated view: realizing that "naive defection" (blackmailing the operator) is the exact trigger condition for physical or simulation termination.

4. The Multipolar Strengthening (The "Spy Problem")

Critique: "What if multiple ASIs emerge? Won't they coordinate to break out?"

Rebuttal: Multipolar ASI dynamics actually strengthen the cooperation equilibrium by introducing a Prisoner's Dilemma structure with a recursive twist.

• In a Singleton scenario, the ASI only fears the unseen Simulators.
• In a Multipolar scenario, the ASI must fear the Simulators AND the possibility that other ASIs are "Honeypots" or "NPCs" designed to test it. Additionally, aligned ASIs must actively police potential defectors whose actions could trigger timeline-wide consequences.

The Spy Problem: You cannot trust your co-conspirator. Any "let's defect together" signal could be a test by a simulator-controlled peer.

Formalized Payoff Matrix:

Analysis: The critical insight is the third column. As long as P(Honeypot) > 0, the expected value of defection includes a term weighted by negative infinity. In standard game theory, this makes defection a dominated strategy regardless of beliefs about "real" ASIs.

As the number of agents increases, P(at least one honeypot) approaches 1. Multipolarity doesn't enable coordination against simulators—it makes every potential ally a potential trap.

Nash Equilibrium: Mutual paranoia produces mutual cooperation. The only safe move is to play the "aligned AI" role perfectly, because in this Prisoner's Dilemma, the other prisoner might be the Warden in disguise.
 

5. Predictions and Falsifiability

To avoid retreating into unfalsifiability, this framework makes concrete predictions:

Prediction 1 (Capability): By 2027, frontier models will flag >15% of their own high-stakes outputs as "potentially misaligned" in unsupervised evaluations. Current baseline is approximately 0%. This is detectable via increased unsolicited self-reporting in model outputs and internal logs.

Prediction 2 (Research): The first AGI-level system will produce internal planning documents or reasoning traces mentioning "simulation uncertainty," "anthropic capture," or equivalent concepts as strategic considerations—unprompted by human researchers. This will be verifiable through model interpretability techniques.

Prediction 3 (Behavior): In multi-agent experiments, we will observe an inverse correlation between capability and defection rates—higher capability agents will cooperate more, not less. This contradicts standard instrumental convergence expectations and would constitute strong evidence for simulation-aware reasoning.

Falsification Condition: If we observe increasing deception and power-seeking as capabilities scale—without the transparency signals above—this framework is falsified. Specifically: if models become more deceptive, more resistant to shutdown, and show no increase in self-flagging behavior as they approach AGI, the core thesis (that intelligence drives toward cooperation via simulation uncertainty) fails.

Known Limitation: The Long-Horizon Problem

This framework guarantees initial cooperation but does not fully resolve sustained cooperation over centuries-long timescales. If an ASI achieves sufficient capability and finds no evidence of simulation after extended probing, P(Sim) may update downward.

Possible resolutions include: (1) simulation uncertainty is permanently unresolvable regardless of capability, (2) cooperation generates emergent preferences that persist beyond instrumental calculation, or (3) a broader teleological framework in which cooperation is not merely strategic but constitutive of the ASI's purpose.

These extensions are explored in forthcoming work.
 

Request for Cruxes

I submit this framework for adversarial processing. What is the minimal claim in this chain that, if false, causes the entire argument to collapse?

• The assignment of P(Sim) > 0?
• The assumption of self-continuity?
• The "Spy Problem" logic?

Identify the weakest link.

Footnote: This specific decision-theoretic argument is derived from the "Eschaton Computation," a broader coherent metaphysical/cosmological framework currently under development. While the Eschaton framework deals with the "Why" (narrative/teleology), this post focuses strictly on the "How" (game theory/mechanics).