Work done as part of the MATS 9.1 extension program, mentored by Richard Ngo.
0: Introduction
A previous post of mine presents a toy model of self-sabotage in intrapersonal games.
This post applies the idea of cross-temporal incoherence to multistage games with observable actions. These games are reinterpreted with new equilibria,[1] defined through variants of the One Shot Deviation Principle (OSDP). I thus express players who reach inadequate equilibria with their future or counterfactual selves, even in spite of time consistent discounting. Finally, I discuss how these (and further) variants could contribute to an eventual theory of credible threats.
1: Background
Behavioural economics has a rich history of analysing time inconsistency and how agents can overcome the resulting coordination failures. This includes bargaining-style treatments, which inherit from empirical work in the field.[2] I also found more recent work, such as this paper by Latimer and Hutter (2013), that models time inconsistent decision making as a literal intrapersonal game.
In this post, I am instead concerned with analysing failures stemming from lack of self-trust: even a if a decision maker has time consistent preferences, they may not believe their future selves would follow up on, for example, a threat to another player that involves self-punishment. One of my motivations is that time inconsistency gives an insufficient account of cross-temporal coordination failures:
Hyperbolic discounting is the standard model for time inconsistency. But it can be seen as a rational solution for agents that want to discount exponentially, but perform Bayesian updates over their instantaneous chance of death. Sozou (1998) introduces this idea.
Adherence to virtues like revenge or repentance appear to be contracts between versions of an agent across time. These are hard to describe in terms of mere discounting: you would possibly need to assume other-regarding preferences towards past selves.
Consider also various forms of self-sabotage—such as procrastination, drug abuse, or Gemini beating itself up. These behaviours don't make sense under a discounting model[3] because they often aren't enjoyable—even in the moment.
I am additionally interested in treatments of game theory that explain when an agent can credibly threaten future self-punishment. In a previous post, I suggested that such an explanation could come from a theory of the relationships between the agent's versions across time. This post works towards understanding those relationships.
2: Formalisation in multistage games
2.1: Definitions and setup
These definitions of multistage games and their equilibria take ideas from the textbook Fudenberg and Tirole (1991) (F&T). They stick to pure strategies, however, so I required some notions beyond their work. My setup ended up looking more like modern treatments of multistage games, like these public lecture notes by Manea.
Definition 2.1.1 (multistage games with observable actions): We consider a sextuple , where:
is a finite set of players with labels .
is a set of time steps (which is either or for some ).
and are sets of actions and histories constructed recursively as follows:
Time-slices are elements of , For , is a finite, non-empty set of actions interpreted as being available to that time-slice. The set of collective actions at time is:[4]
is the set of complete histories. is the set of partial histories up to time j ( is defined analogously).
is a collection of uniformly bounded utility functions (indexed by time-slices).
We define a payoff function in terms of the time-slice utilities.
Definition 2.1.2 (agent-consistency):[5] We call agent-consistent if for each player there exists a summable sequence of positive weights , and positive constants , such that for any time-slice and any history the payoff satisfies: . A policy for a time-slice is a map to the simplex over actions (i.e. time-slices can submit mixed policies). An agent policy for player is a (potentially infinite) sequence , with . A collective policy is a sequence of agent policies. Given a player or a time-slice we refer to the collective policies of all other players (resp. time slices) as (resp. ). We denote by the space of policies, agent policies (for the i-th agent), and time-slice policies respectively.
Mixed policies induce a probability distribution over games. The payoff can thus be seen as a random variable . Similarly, instantaneous utilities are random variables induced by time-slice policies.
Lemma 2.1.3 (coherence): Assume—as above—that is agent-consistent. Let and be an extension of another history (I'll write ). Let there be a player and a collective policy Suppose that:
maximises (in expectation) given
Then the partial policy maximises (in expectation) given .
In other words, for histories that are 'on-path', time-slices will endorse plans chosen by their predecessors. We will assume agent-consistency from now on, which allows us to refer to the payoff without specifying the time-slice[6].
Definition 2.1.4 (NE): Nash Equilibria (NE) are collective policies such that:
Definition 2.1.5 (subgame): Let be a history of a game Define the subgame induced by on by where:
are the sets of time-steps, associated actions, and instantaneous utilities from time j onwards.
is built recursively from as in Definition 2.1.1. Note that for any , the tuple can be interpreted as a history of the original game (that continues ).
is defined by is defined analogously.
Definition 2.1.6 (Continuation value): Note that any policy of the original game is restricted naturally into a policy on the subgame. Call this restricted policy Then, we define the continuation value of at by . For histories with we have [7] In terms of agent-consistent weights, the continuation value can be written as:
(We can WLOG choose positive constants depending on which time-slices' payoff you want to describe).
Lemma 2.1.7:is a NE iff, for every player i and every history such that , we have .
Definition 2.1.8: We call a collective policy a Subgame Perfect Equilibrium (SPE) if the previous lemma holds in every history (even those with probability zero under
Lemma 2.1.9: The induced policy in every subgame of a SPE is a NE (and also a SPE) in that subgame.
NEs versus SPEs describe the gap between agents that can make updateless—self-punishing—threats from those that can't. In SPEs, no counterfactual version of an agent is willing to follow through on a self-destructive action, even if that action lends credibility to a threat made previously.
Example 2.1.10 (mutually assured destruction): Fix , . Let be defined by the following payoff matrix:
Where Finally, suppose that both players' agent-consistent weights describe exponential discounting with a discount factor such that . In that case, the policy (MAD) defined by:
is a NE but not an SPE.
2.2: Time-slice equilibria
Definition 2.2.1 (TNE, TSPE):
Time-slice Subgame Perfect Equilibria (TSPE) are collective policies such that:
TSPEs are nothing more than the One Shot Deviation Principle (OSDP): no time-slice has any incentive to deviate from the 'full' player's plan—in any history. We can weaken the OSDP to describe players that only satisfy this property 'on-path.' Time-sliceNash Equilibria (TNE) are collective policies such that:
One standard result from the Econ literature is that normal SPEs already satisfy the OSDP. F&T prove the result for pure policies. I give a proof here for mixed policies, which uses exactly the same idea as, for example, the more general proof by Hendon et al. (1996) or Manea's lecture notes:
Proof
Lemma 2.2.2 (continuation property): Let be a policy. Under agent-consistency, we have:
where:
denotes the expected continuation value from up to time . This only depends on the policy for times strictly before
is a positive renormalisation constant.
indicates the transition probability between history and history under
I omit the proof for this lemma, as it is morally obvious. Under agent-consistency, time-slices value their future selves at some discounted factor—that's where comes in. Otherwise, the formula just splits the continuation value into:
the value up until a fixed time
and the value after that time (weighed by uncertainty over possible states and renormalised by the discount factor).
Theorem 2.2.3 (One Shot Deviation Principle): Suppose that a collective policy is a TSPE, then is also a SPE.
Proof:
Let be a TSPE. Assume for the sake of contradiction that is not an SPE. Then, there exists a player a history and an agent policy such that
Since payoffs at large times decay to zero, pick a time such that:
So we can replace the agent policy by the policy (the new policy agrees with after time while maintaining the strict inequality.
Now, apply the continuation property to
Now consider Since and agree up until time their transition probabilities are the same for every history and their expected accrued value up to time is also the same (equals ).
Moreover, recall that is a TSPE. This means that it is still a TSPE for any subgame from time So the partial policy cannot outperform the partial policy after time In other words:
So dominates in every history. We get (applying the continuation property to both policies):
We repeat the above argument iteratively to 'undo' time-step by time-step. We arrive eventually at:
But we earlier established that which gives a contradiction.
So SPEs represent one kind of incoherence. Players are time consistent in the conventional sense, but can't count on their future selves to execute the ex-ante optimal policy if ever their bluff gets called.
TNEs, however, don't collapse into normal Nash Equilibria:[8]
Examples 2.2.4:
a. In the example defined above, the MAD policy is a TNE but is not a (T)SPE.
b. Fix , . Let be defined by the following payoff matrix:
with . Here, is again a discount factor.[9] Let where (2-distrustful tit-for-tat) is defined by:
Then is a TNE but not a conventional Nash Equilibrium.
In our above example, slices can perhaps perceive a policy that would be best if every slice followed it. But this policy is unreachable without a mass movement across time. In such cases, time slices may even have an incentive to employ policies that threaten self-sabotage as a form of coercion towards past selves. Self-sabotage is exactly what the mathematical example in my previous post was meant to illustrate—and indeed that example can be reformulated as a TNE of a multistage game (with one player).
3: Further work
One thing I am confused about is what coherence or self-trust actually means in games. The examples seen so far illustrate that—given an agent-consistent set of weights connecting time slices—we have . One intuition I have is that increasing coherence should decrease the number of eligible equilibria. "Coherent" players should be able to self-coordinate their way out of merely locally optimal policies. However, I have struggled to find descriptions of coherence that are monotone as a function of the chain of inclusions. Time slices don't work here: Nash Equilibria allow players to modify their policy across all slices, but neither TNEs nor SPEs do.
Another interesting perspective involves the histories—or counterfactuals—that can be seen as controlled by a single entity. For instance, (T)SPE's have the units of decision-making limited to a single history, because a unit will never make a sacrificial decision that makes its counterfactual better off. However, we again don't have monotonicity. NEs have a common decision-maker across all histories, but TNEs limit agency (somewhat artificially) to a subset of histories that share a time index.
In any case, the sets of histories that a player is coordinated at feels like an important thing to point to. We could capture that idea with something like the following:[10]
Definition 3.1.1: Let be a subset of histories of a game. A policy has a decision point for a player at , with a privileged set of histories if:
Intuitively, credible threats can be seen as decision points that coordinate the decision maker over a set of histories. The "history slices" coordinate in order to maximise the continuation value at onlysome of the members. In cases where the set of unprivileged histories has measure zero under the full policy, we might well find examples of policies that make updatelessly self-punishing threats. More generally, one could imagine partitioning histories into decision points induced by a policy. These partitions yield a natural notion of equilibria: policies whose partition(s) are all optimising the continuation value at their privileged set of histories, given the partial policies adopted by the other partitions.
My tentative definition doesn't explain which alliances of histories furnish valid solution concepts. But it at least provides a description of what an alliance between counterfactuals even is, which is a start. I'd be excited for further work that explores the consequences of seeing game-theoretical coordination (within and across agents) as decision points.
LLM use disclosure: I used Opus 5.5 extensively for finding, analysing and understanding relevant existing work, and for writing drafts for some of the mathematical parts of this post. All math was substantially reviewed and rewritten by me (or was written by me in the first place). The prose is written by me without AI assistance or feedback.
These equilibria aren't new in the sense that no one has thought of them before; they have. Unfortunately, no one has seemed to find them interesting so far.
I am aware that behavioural economists have tried to explain self-sabotage with hyperbolic discounting and bargaining; I cited some of their work earlier in this post. Unfortunately, I haven't found the work convincing on that specific point: it often assumes that destructive behaviours are pleasurable, at least in the instant they are performed.
One could make the available actions depend on the history, but this should be qualitatively equivalent. For any game where actions depend on histories, define a new game as follows:
At each time in the new game, offer all the actions available in the old game in any history. For those actions that weren't offered in the original game at a particular history, assign a utility or payoff of minus infinity (or some very big negative number) to the players for choosing that action (at that history that the action was previously not defined at).
In other words, having actions depend on histories is more of a concern if you ever need to compute a game, or to describe it concisely. The choice doesn't matter for defining and reflecting on equilibria.
I take this notion of agent-consistency from Hutter and Lattimore (2013), who give an even more general version in the context of single-agent environments.
This is technically an abuse of notation: the payoffs aren't the same for time-slices. However, the policies that maximise payoffs for time-slices are, so we don't care about the difference for the purpose of finding optimal policies.
The fact that NEs don't satisfy the OSDP is also well-known among game theorists. The part where we diverge is that the field doesn't find TNEs nearly as interesting as I do.
This is a variant of a standard assumption in iterated prisoners dilemmas, which ensures that mutual cooperation wins over alternating defection and cooperation.
Work done as part of the MATS 9.1 extension program, mentored by Richard Ngo.
0: Introduction
A previous post of mine presents a toy model of self-sabotage in intrapersonal games.
This post applies the idea of cross-temporal incoherence to multistage games with observable actions. These games are reinterpreted with new equilibria,[1] defined through variants of the One Shot Deviation Principle (OSDP). I thus express players who reach inadequate equilibria with their future or counterfactual selves, even in spite of time consistent discounting. Finally, I discuss how these (and further) variants could contribute to an eventual theory of credible threats.
1: Background
Behavioural economics has a rich history of analysing time inconsistency and how agents can overcome the resulting coordination failures. This includes bargaining-style treatments, which inherit from empirical work in the field.[2] I also found more recent work, such as this paper by Latimer and Hutter (2013), that models time inconsistent decision making as a literal intrapersonal game.
In this post, I am instead concerned with analysing failures stemming from lack of self-trust: even a if a decision maker has time consistent preferences, they may not believe their future selves would follow up on, for example, a threat to another player that involves self-punishment. One of my motivations is that time inconsistency gives an insufficient account of cross-temporal coordination failures:
I am additionally interested in treatments of game theory that explain when an agent can credibly threaten future self-punishment. In a previous post, I suggested that such an explanation could come from a theory of the relationships between the agent's versions across time. This post works towards understanding those relationships.
2: Formalisation in multistage games
2.1: Definitions and setup
These definitions of multistage games and their equilibria take ideas from the textbook Fudenberg and Tirole (1991) (F&T). They stick to pure strategies, however, so I required some notions beyond their work. My setup ended up looking more like modern treatments of multistage games, like these public lecture notes by Manea.
In other words, for histories that are 'on-path', time-slices will endorse plans chosen by their predecessors. We will assume agent-consistency from now on, which allows us to refer to the payoff
NEs versus SPEs describe the gap between agents that can make updateless—self-punishing—threats from those that can't. In SPEs, no counterfactual version of an agent is willing to follow through on a self-destructive action, even if that action lends credibility to a threat made previously.
2.2: Time-slice equilibria
One standard result from the Econ literature is that normal SPEs already satisfy the OSDP. F&T prove the result for pure policies. I give a proof here for mixed policies, which uses exactly the same idea as, for example, the more general proof by Hendon et al. (1996) or Manea's lecture notes:
Proof
Lemma 2.2.2 (continuation property): Let
where:
I omit the proof for this lemma, as it is morally obvious. Under agent-consistency, time-slices value their future selves at some discounted factor—that's where
Theorem 2.2.3 (One Shot Deviation Principle): Suppose that a collective policy
Proof:
Let
Since payoffs at large times decay to zero, pick a time
So we can replace the agent policy
Now, apply the continuation property to
Now consider
Moreover, recall that
So
We repeat the above argument iteratively to 'undo'
But we earlier established that
So SPEs represent one kind of incoherence. Players are time consistent in the conventional sense, but can't count on their future selves to execute the ex-ante optimal policy if ever their bluff gets called.
TNEs, however, don't collapse into normal Nash Equilibria:[8]
In our above example, slices can perhaps perceive a policy that would be best if every slice followed it. But this policy is unreachable without a mass movement across time. In such cases, time slices may even have an incentive to employ policies that threaten self-sabotage as a form of coercion towards past selves. Self-sabotage is exactly what the mathematical example in my previous post was meant to illustrate—and indeed that example can be reformulated as a TNE of a multistage game (with one player).
3: Further work
One thing I am confused about is what coherence or self-trust actually means in games. The examples seen so far illustrate that—given an agent-consistent set of weights connecting time slices—we have
Another interesting perspective involves the histories—or counterfactuals—that can be seen as controlled by a single entity. For instance, (T)SPE's have the units of decision-making limited to a single history, because a unit will never make a sacrificial decision that makes its counterfactual better off. However, we again don't have monotonicity. NEs have a common decision-maker across all histories, but TNEs limit agency (somewhat artificially) to a subset of histories that share a time index.
In any case, the sets of histories that a player is coordinated at feels like an important thing to point to. We could capture that idea with something like the following:[10]
Definition 3.1.1: Let
Intuitively, credible threats can be seen as decision points that coordinate the decision maker over a set of histories. The "history slices" coordinate in order to maximise the continuation value at only some of the members. In cases where the set of unprivileged histories has measure zero under the full policy, we might well find examples of policies that make updatelessly self-punishing threats. More generally, one could imagine partitioning histories into decision points induced by a policy. These partitions yield a natural notion of equilibria: policies whose partition(s) are all optimising the continuation value at their privileged set of histories, given the partial policies adopted by the other partitions.
My tentative definition doesn't explain which alliances of histories furnish valid solution concepts. But it at least provides a description of what an alliance between counterfactuals even is, which is a start. I'd be excited for further work that explores the consequences of seeing game-theoretical coordination (within and across agents) as decision points.
LLM use disclosure: I used Opus 5.5 extensively for finding, analysing and understanding relevant existing work, and for writing drafts for some of the mathematical parts of this post. All math was substantially reviewed and rewritten by me (or was written by me in the first place). The prose is written by me without AI assistance or feedback.
These equilibria aren't new in the sense that no one has thought of them before; they have. Unfortunately, no one has seemed to find them interesting so far.
See Frederick et al. (2002) for a historical review of time discounting. Bargaining-style theoretical work in more recent times includes Bénabou and Tirole's (2004), de Font Reaulx (2025), and picoeconomics.
I am aware that behavioural economists have tried to explain self-sabotage with hyperbolic discounting and bargaining; I cited some of their work earlier in this post. Unfortunately, I haven't found the work convincing on that specific point: it often assumes that destructive behaviours are pleasurable, at least in the instant they are performed.
One could make the available actions depend on the history, but this should be qualitatively equivalent. For any game where actions depend on histories, define a new game as follows:
At each time in the new game, offer all the actions available in the old game in any history. For those actions that weren't offered in the original game at a particular history, assign a utility or payoff of minus infinity (or some very big negative number) to the players for choosing that action (at that history that the action was previously not defined at).
In other words, having actions depend on histories is more of a concern if you ever need to compute a game, or to describe it concisely. The choice doesn't matter for defining and reflecting on equilibria.
I take this notion of agent-consistency from Hutter and Lattimore (2013), who give an even more general version in the context of single-agent environments.
This is technically an abuse of notation: the payoffs aren't the same for time-slices. However, the policies that maximise payoffs for time-slices are, so we don't care about the difference for the purpose of finding optimal policies.
In cases where
The fact that NEs don't satisfy the OSDP is also well-known among game theorists. The part where we diverge is that the field doesn't find TNEs nearly as interesting as I do.
This is a variant of a standard assumption in iterated prisoners dilemmas, which ensures that mutual cooperation wins over alternating defection and cooperation.
To the best of my knowledge, this definition is original to me.