TL;DR. LessWrong's decision-theory debates (Newcomb, FDT vs CDT, counterfactual muggings) are almost entirely about what we suppose when we consider a candidate action or policy. There is a second, older, semi-orthogonal, but not fully orthogonal question: how to score a gamble once you know the possible outcomes. The predominant (and mostly implicit) answer to that was "take the expected utility". This post treats the two questions, plus a question about how a choice made before receiving information should relate to choices made afterward, as separate axes, and maps every decision theory you have heard of (and several nobody has built) into the resulting grid. Interestingly, the axes are provably entangled: theorems old and new show that there exist different restrictions on what places in that "decision-theoretic space" are inhabitable. I think there is a structure, maybe a deep and consequential structure, inside this map of decision theories which shows what possible combinations across the axes are coherent and fruitful. If we study it, we may understand the entire set of all possible coherent decision theories, something akin to the "metatheory of decision theories". It may be useful to know the entire set.
This is both a self-educational note and a research post. I have tried to state the scope of every result. While I am not certain about every conclusion about the "space of decision theories," I am confident that thinking in terms of the proposed space is useful. In the most conservative case, it is a nice map of decision theories. In a more optimistic case, the space has non-trivial structure whose study may lead to new results. Note also that this is a rather technical post. I tried to make it as simple as I could, but at some point, you lose precision, and I didn't want to go further that route, because it is meant not only as an educational post, but also as a research one. That said, I think with some additional effort it may become more clear. In particular, I added many footnotes that clarify terms (without putting a lot of theory in the body itself), and I think more footnotes can be added. Most of the footnotes are AI-generated, prompted by me to focus on addressing some common cases of confusion. Please let me know if something should be clarified. Also, as the post sometimes goes quite technical and is rather long, feel free to skip some pieces which are boring to you. I'd rather have you read intro and final sections (7 and 8) than read only the first half of the post.
Technical appendix — formal statements, proof sketches, finite numerical examples, and open problems (for now not verified, many things AI-generated, comes with no guarantee!): appendix link.
"This specifically assumes that expected utility maximization is the right way to deal with mathematical uncertainty. Consider it a temporary placeholder until that problem is solved."
This post began as a sequel to On The Independence Axiom. That post argued that the independence axiom is stronger than rational choice requires.[1] Independence is enough, as part of a larger set of assumptions, to support expected utility. But I argued that rejecting independence does not by itself entail inconsistency.
The discussions that followed, both online and offline, taught me something about how LessWrong talks about decision theory. When I asked how we should evaluate a probability distribution over outcomes, people often answered with ideas about which distribution should be used or from which informational point a policy should be chosen: FDT, UDT, updatelessness, and counterfactual mugging. This is not bad, per se. Some of the responses were especially helpful, including Wei Dai's reference to his 2009 post; so was Scott Garrabrant's closely related point about updating, made in another discussion. But the responses were often answering a different question from the one I had asked.
I therefore wanted to clarify the separation of three questions.
Question 1: How should we evaluate a prospect?
Suppose each available action has already been matched with a prospect[2]—in the simplest case, a probability distribution over outcomes. How should we rank those prospects?
Expected utility theory says: give each outcome a utility, multiply each utility by its probability, and add the results.[3] This rule follows from the standard von Neumann–Morgenstern assumptions taken together: a weak ordering of options, suitable regularity or continuity, and independence. It does not follow from independence alone.[4]
Question 2: Which prospect should be used for each candidate?
At the first rough glance, it is akin to the question of “what would happen if I chose this action?”, but this is too ambiguous. It can mean at least three things:
What should I expect after learning that I choose the action?
What would change if the action were causally set to that value?
What else would be different if the decision procedure produced that output?
These are not simply three forecasts about what physically happens after the agent moves. Two theories can agree on the full causal and statistical model, including which physical effects each act causes, and still use different probability distributions in deliberation. They differ over which hypothetical comparison is relevant to choice. Advocates of the theories may also disagree about the background model, but that is a separate disagreement (for example, a disagreement may be whether the predictor is running a physical simulation of you or is instead a crude statistician relying on a correlation); the supposition-rule distinction remains even when the model is held fixed.
I will call the rule for making that comparison the supposition rule.[6] Given a background model and a candidate action or policy, it says:
what event, act, or decision variable is being supposed different;
which other parts of the model are held fixed; and
which parts are allowed to vary with the supposed decision.
The uncertainty representation that results from this operation (usually a probability distribution) is the candidate's decision-relevant prospect. A prospect is therefore not attached to the bare physical act alone, but is produced by applying a supposition rule to that act or policy within a model.
With that in mind, the main theories can now be stated more precisely:
Evidential decision theory (EDT) supposes that the agent learns “I choose ” and conditions on that event. Anything statistically associated with choosing , including an earlier common cause or prediction, can then receive a different probability.
In a standard interventionist presentation, causal decision theory (CDT) supposes that the relevant act or decision variable is causally set to . Variables causally upstream of that intervention keep the distribution supplied by the agent's current information and causal model; effects downstream of it can change. Other CDT formulations use causal counterfactuals or dependency hypotheses rather than literal structural interventions, but preserve the same causal-versus-evidential thing.
Functional decision theory (FDT) supposes a different output of the relevant decision function and allows represented facts that depend logically or subjunctively on that output (such as a predictor's prediction or the output of an identical copy) to vary with it.[7]
For example, consider a version of smoking lesion in which a lesion causes both the choice to smoke and cancer, smoking itself does not cause cancer, and the agent's present evidence does not already screen off the correlation between its act and the lesion.[8] EDT then treats smoking as evidence of the lesion and evaluates it using the corresponding conditional distribution. CDT intervenes on smoking, leaves the probability of the earlier lesion unchanged, and therefore uses a different prospect. The disagreement is not over whether smoking physically causes cancer, but over which distribution is relevant when comparing the acts.
Question 3: How should choices at different times fit together?
An agent may choose before learning something, and then choose again after learning it. How should these choices be related?
A naive updateful agent solves the problem again at each later decision point and, when planning earlier, does not fully account for a later preference reversal.
A sophisticated agent, following Strotz's consistent-planning idea, predicts what its future selves will choose and makes its earlier choice with that prediction in mind.
A resolute agent, in McClennen's terminology, updates its beliefs but carries out a plan chosen in advance.
An updateless agent chooses a policy from a specified pre-observation point of view. The policy itself can still tell the agent to respond differently to different observations.[9]
Resolute and updateless agents may behave alike in some decision trees, but the ideas are not the same. Resoluteness describes an agent's relation to an earlier plan. Updatelessness describes the point of view from which the policy-selection problem is set up.
The standard von Neumann–Morgenstern representation of preferences over lotteries does not by itself answer Questions 2 or 3. It takes lotteries as the starting objects and does not specify how an act generates a lottery or how later choices should relate to an earlier plan. And much LessWrong work holds the answer to Question 1 fixed at expected utility and changes the answer to Question 2, Question 3, or both. The FDT paper says this directly:
All three of EDT, CDT, and FDT are expected utility theories, meaning that they prescribe maximizing expected utility. (Yudkowsky & Soares 2017, §5)
The same paper approvingly quotes James Joyce's 1999 claim that rival decision theories need not offer rival theories of value; they may instead disagree about the point of view from which actions should be evaluated. UDT likewise used expected utility over logical or mathematical uncertainty as a placeholder rather than deriving it. That leaves open the question of how other evaluation rules combine with different supposition rules and different ways of handling later choices.
The division itself is not historically new. Joyce made part of it explicit, the FDT paper quotes him, and Lara Buchak's survey of decision theory covers several of the same distinctions. Still, it is worth making the division more visible. I repeatedly encountered confusion that would have been easier to avoid if the three questions had been stated separately.
However, this post is not about the division itself.
What I want to do here is to cross all these dimensions, and map the entire zoo of decision theories into the resulting space: occupied cells, promising cells, open cells, and cells that are provably closed. Or, to be more precise, to place familiar theories in this three-part space and state the known compatibility results together with their assumptions.
Here is the central claim:
The three questions are different in kind, but their answers are linked by conditional requirements of coherence.[10] Almost any combination of answers can be written down and used to compute a recommendation. But the constraints appear as soon as you ask the resulting agent to behave well in some specific way. Each such demand is really the avoidance of a specific failure, like for example not being exploitable or not contradicting yourself over time. Each of these demands is the subject of at least one theorem, and every one of those theorems has premises, of course. In the three-dimensional space of decision theory, there are "islands of stability", and they are, to a degree, regular.
Two immediate clarifying notes, however:
First, whether a combination is acceptable is not a property of the combination by itself. It depends on which demands you place on the agent and which environments you expect it to face. So the useful output I am trying to achieve is not a verdict stamped on each combination, but a record of the form: this bundle of answers, in this class of environments, satisfies this demand, under these assumptions, at this price. Some prices are substantial, like committing in advance to a plan you may later wish to revise, restricting which beliefs you are allowed to hold, or changing how you update, and naming the price is informative by itself. Note that initially I tried to find rather "universally consistent cells", but that simply didn't work. Maybe it's just me failing. But for all I see, all consistency results are very conditional.
Second, the failures are not scattered at random. They cluster around a small number of structural features that recur from theorem to theorem: how the agent treats branches it did not end up on, whether it re-solves the problem from scratch after learning something, and whether the environment responds to its whole plan rather than only to its actions. That recurrence is the reason I suspect this space has real interesting structure, rather than being a list of unrelated special cases.
So, the idea is that "is this way of scoring gambles coherent?" is not, by itself, a well-posed question. Coherence belongs to the whole bundle: how you score gambles, what you suppose when you weigh a candidate action, and how today's plan constrains tomorrow's choice. Demand a strong enough form of consistency across decision trees, and you are pushed to an expected-utility representation at each information event. Keep a rank-dependent rule instead and re-apply it as information arrives, and your later self can overturn the plan your earlier self preferred. Take one of the standard repairs (commit in advance to the plan, or restrict your beliefs so the conflict cannot arise), and you have bought protection against one named failure, on that repair's own terms and no further. Rejecting expected utility is therefore not automatically incoherent, but it is a commitment about the whole bundle.
Brian Weatherson's Four Problems in Decision Theory and Game Theory as Decision Theory are the closest neighbors to this post. His derivations from the Single Choice Principle (SCP)[11] are strong examples of the kind of compatibility result between the three axes I have in mind. For example, he argues that SCP forces Buchak's risk function to be the ordinary linear one, and that SCP plus further assumptions leads to a form of causal ratificationism.[12] This entire line of reasoning anticipates what I am trying to say, but more generally.
I do have some disagreements, but feel free to skip this paragraph as it is not super essential for the post. However, if you want to hear: In problems where the agent faces only one real choice, SCP demands that planning and doing agree: if it is rational to decide in advance "should I reach this point, do X," then on reaching that point, X must be what it is rational to do. The motivation is a Ramsey-style test, which treats two questions as one — "what should I do if p turns out to be true?" and "p is true; what should I do?" That identification is precisely what resolute theorists, and theorists with global, nonseparable preferences,[13] deny, and I think for a good reason: on their view the earlier question concerns a whole gamble, part of whose value lies in branches that will never be reached, while the later question concerns only a fragment of it. There are two different objects, so agreement between the answers is not automatic. SCP is therefore a compatibility result of the same conditional kind as those which will be described in §5, and Weatherson himself notes that violating it produces no money pump or sure loss, only an alleged inconsistency.
2. A compact formal picture (almost)
Write for the background model, which is everything the theory takes as given about the situation: which acts are possible, what causes what, what the agent believes, what a predictor knows. A one-shot decision problem then looks like this:
In words, it is something like this: out of the plans available to you, pick the one whose associated prospect scores highest. Before that instruction means anything, three things must be supplied: the available plans, the rule turning a plan into a prospect, and the rule scoring prospects (and a fourth is needed as soon as the problem unfolds over time). The list of available plans and the space of prospects are themselves part of what a model must specify, and they have to fit together: whatever hands over must be the kind of object knows how to score.
— the plans you may choose among. The model must say which actions are available at each point the agent might reach, and which complete plans count as available. What is usually called a pure policy fixes one action for every situation the agent might find itself in, where a "situation" means a possible history of observations; a behavioral policy tosses a fresh coin at each point, while a mixed policy tosses one coin at the outset to select an entire plan, which allows the choices made at different points to be correlated.[14] A single one-shot act is the simplest case of all — a plan with one entry. One warning, however, since a great deal of definitional discussion has turned on it: describing candidates as plans is a matter of notation. It does not imply that the agent has committed to anything in advance, or that its plan was somehow fixed before the environment, the predictor, or the observations came into being.
— the rule that turns a plan into a prospect. This is Question 2 made formal. Give it a candidate plan and it returns the prospect that the theory will actually score. In the ordinary case a prospect is just a probability distribution over possible outcomes or whole trajectories. A model with several candidate probability distributions, a two-level model that keeps two kinds of uncertainty apart, or a model that keeps the act-and-state structure visible instead of collapsing it may need a richer object. What the rule has to record is what is supposed to be different, what is held fixed, and what is allowed to move along with the supposition, including, when the plan involves randomizing, when the coin is tossed and who gets to see it. Each theory owes its own version. EDT conditions on the event "I choose this". A causal theory must say exactly what is being intervened on (the act, the decision, a disposition, or an earlier choice of policy) and what counts as lying downstream of that intervention, and so is allowed to change. FDT owes an account of which facts vary with the output of the decision procedure, whether logically or subjunctively. The fact that these theories belong in one slot seems intuitive, but formally, Joyce's representation theorem is what licenses treating this as a single slot: he showed that causal and evidential decision theory (as presented usually) are the same expected-utility formula with different supposition rules plugged in. (FDT's authors claim their theory meets his constraints too.) However, what the theorem does not do is tell you how to use such a rule when decisions come in sequence: what to suppose at a later point, whether to apply the rule afresh to the remaining problem, or how supposing interacts with what the agent has learned in the meantime. That is the work of below.[15]
— the rule that scores a prospect. This is Question 1 made formal: it takes a prospect and returns one real number. Expected utility sets : average the utilities, weighting by probability. Rank-dependent utility instead reweights cumulative probabilities, so how much an outcome contributes depends partly on where it ranks in the gamble. Buchak's REU is pinned down by three ingredients — a probability, a utility function, and a risk function . Garrabrant's geometric rationality averages geometrically rather than arithmetically, including two-level forms that first take an ordinary expected value inside each component and then a geometric mean across components; depending on the model, those components may be possible worlds, copies of the agent, or locations.
— what I call "the rule relating choices made at different times". This is Question 3 made formal, but it is the part the formula above cannot capture. That formula answers "what should I do?" at one decision point. But an agent working through a problem reaches many such points, and asking the question afresh at each one does not by itself produce a coherent course of action, because the answers can conflict, and something must say which of them the agent actually follows. That is what supplies. Filling it in means answering several separate questions, as I see it. When planning, does the agent take into account what its own later selves will do? From which standpoint is a policy chosen: before the agent has seen anything, or after? When information arrives, does it solve the problem again? And if an earlier plan and a later preference disagree, which one governs what happens? These are not four settings on a single dial. An agent can choose its policy from the pre-observation standpoint and still carry it out resolutely, and the four labels in §1 combine these features in different ways. Sophistication answers the first: the sophisticated agent forecasts what its later selves will do, and it does this precisely because it expects those later selves to get their way, so anticipation is a way of living with re-optimization rather than an alternative to it. Updatelessness answers the second: it fixes the standpoint from which a policy is chosen, and by itself says nothing about what happens if the agent later disagrees. Resoluteness answers the last: the earlier plan governs, whatever the later self would now prefer. Since these are different questions, their answers combine: an agent can select a whole policy from the pre-observation standpoint and then execute it rather than re-deriving it, which is one coherent design rather than a contradiction. It also means the labels can match in behavior without matching in content: as §1 notes, resolute and updateless agents may behave alike in some decision trees while answering different questions.
A theory that runs over time also has to say how an observation the agent actually makes changes its beliefs or its model. Call that its learning rule . It is easily confused with , so the difference is worth stating: builds a hypothetical — "suppose I were to do this" (for the purpose of comparing candidates), whereas responds to something that really happened. Ordinary Bayesian models bury the learning rule inside by assuming conditionalization; Nielsen's framework and several ambiguity models deliberately vary it instead. So the three headline questions are not a complete inventory of a theory's formal inputs. Wherever learning is not already fixed by the background model, the learning rule has to be written down separately.
Some theories need more structure still. For example, a ratification rule asks whether a contemplated choice still looks acceptable once the agent supposes it is about to make it. It is like a stability test applied inside a single deliberation, which is not automatically a rule about how choices at different times should relate. A theory with incomplete preferences may return a set of permissible options rather than a single best one. A menu-dependent theory cannot score an option without knowing what else is on offer, which breaks the assumption this section opened with. Where such features matter, the core list , together with any separate learning rule, should be extended.
Maybe, even having covered all that, I am still missing something, but to me it looks like a complete characterization of any decision theory. Still, we will focus only on the three questions stated at the beginning of this post.
3. The map of decision theories
The entire framework, as we saw, has three dimensions, so one flat table cannot display all three independently. The most useful two-dimensional view, as I think about it, puts evaluation and uncertainty families in the columns and decision setups, dynamic packages, or cross-tree constraints in the rows. This preserves comparison by dimension, but it is not a Cartesian grid[16] whose rows and columns are all values of one mathematical type. The following section then lists the components of partially and fully specified families more explicitly.
3.1 Two-dimensional classification
The table below crosses two main things I am trying to compare. Each column is a family of scoring rules: an answer to Question 1. Each row is a decision setup: a way of supposing a candidate action, together with a way of relating earlier and later choices: Questions 2 and 3 packaged as they actually usually appear in the literature. Each cell says what is known about putting that scoring rule together with that setup — a combination that amounts to a concrete decision theory.
I assign the cells one of the five statuses:
Named — the literature contains a reasonably developed proposal. This is not a claim that the proposal has been proved coherent in every relevant sense.
Local — the combination yields a well-defined calculation in at least one problem. This is not a claim of a complete theory of later choices or of self-reflection.
△ — a known difficulty for the familiar way of implementing this combination, such as reversing a choice after re-applying the rule to new information. Generally, not an unconditional impossibility result.
✕ — ruled out, but only given the assumptions of that row. It appears in the first row only. Drop one of those assumptions and the cell can reopen.
Open — I found neither a developed theory nor an impossibility proof. This is a statement about my search, not evidence that the combination works.
The columns.
Expected utility (EU) — this is very clear. Just score a prospect by its probability-weighted average utility.
Rank-dependent (RDU, and Buchak's REU) — reweight cumulative probabilities, so how much an outcome contributes depends partly on where it ranks within the gamble.
Two-level and other global non-EU — rules that keep two kinds of uncertainty apart (ordinary chance inside each possible world; weights across worlds or copies), or that otherwise score a gamble as a whole rather than outcome by outcome.
Growth-rate rules (ergodicity economics, EE)[17] — not one scoring rule but a family indexed by the environment: rank by expected wealth under additive dynamics, by expected log wealth under multiplicative dynamics, and so on. Inside any one fixed environment this coincides with expected utility.
Ambiguity (multiple priors and relatives) — rules for uncertainty about the probabilities themselves. Multiple-prior models are one important representation of ambiguity, not its definition.
A note on the rows. They are packages that occur in the literature, and not (necessarily) mutually exclusive settings. The first row is different in kind from the rest: it is not a decision setup an agent might adopt, but a constraint: a set of demands about choosing consistently across decision trees, which then rules certain scoring rules out.
Decision setup (rows) / scoring rule (columns)
Expected utility
Rank-dependent (RDU, REU)
Two-level / global non-EU
Growth-rate (EE)
Ambiguity (multiple priors)
Hammond's cross-tree requirement — a constraint, not a setup [note a]
Required, in the form of an EU representation at each information event; one common utility across events needs further assumptions
✕ if rank-dependent
✕ if non-EU
Compatible inside each fixed environment; the cross-environment question changes which family is under discussion
✕ if it violates independence
Causal supposition; re-solve at each decision point
Named: classical CDT
Local: a causally generated prospect can be scored by REU (§4.1); REU does not itself select causal supposition
Local: the formula applies, but supplies no theory of later choices
Local, overlapping with EU: Kelly-style multiplicative EE is log-EU inside the fixed environment
Not classified: the ambiguity models cited take no stand on causal vs. evidential
Named but △: these models use acts and states; naively re-applying the rule to each new situation runs into §5.2
Open beyond constructed models
Local: an environment-specific EE rule can be added, but does not fix the updating rule
Named, with restrictions:Epstein–Schneider get consistency by restricting the prior sets[19]; Hanany–Klibanoff get it by letting updating depend on the choice problem; unrestricted maxmin has dynamic problems[20]
Evidential supposition; re-solve at each decision point
Named: Jeffrey-style EDT
Local but △: the calculation is well defined; a full dynamic theory must still handle reversal and information problems
Open beyond local examples
Local, overlapping with EU: an evidential growth-rate rule can be defined separately in each environment
Open; the standard dynamic problems remain
Logical or subjunctive supposition (FDT-style), responsive to observations
Named: FDT proposes this supposition rule, but its one-shot rule does not by itself fix a complete theory of later choices
Local: the FDT-style supposition plus REU calculation in §4.1 is not a full dynamic or self-reflective theory
Open
Open; locally EU once a canonical dynamic is fixed
Open
Sophisticated planning (any supposition rule)
Named: Strotz-style consistent planning
Studied but △: an equilibrium among one's future selves exists in some models, but does not by itself remove information aversion or self-recommendation problems
Constructed: consistent-planning responses; Machina's separate global route appears in §3.2
Locally EU; the cross-environment version is largely open
Resolute: carry out the plan chosen earlier (causal or evidential)
Named: resolute complete-plan choice; agrees with EU re-optimization on positive-probability histories under the conditions in [note b]
Named: the McClennen-style resolute response; Theorem 7 gives only the finite-tree dominance guarantee stated in §5.2
Named: global and nonseparable resolute approaches
Local: you can score whole plans by growth rate and then execute the best one, but nothing in the growth-rate rule itself argues for resoluteness by itself — it answers Question 1 only and leaves the dynamic setup to be supplied separately.
Open in general; related models with plan-dependent updating exist
Updateless: choose the whole policy in advance (logical or policy-based)
Local: §4.2's calculation is EE only under the stipulated repeated multiplicative reading [note c]; the cross-environment distinction is in §8.1
A related family, not a classification of the row: infra-Bayesian approaches include policy-based ambiguity proposals, but their supposition and dynamic rules vary[21]
Notes on particular cells. Read them only if you want to know exactly what the names in the table mean; they are not essential for general understanding.
(a) Hammond's requirement: fix a nonempty finite set of states; consider every structurally admissible finite decision tree built over that setup, with strictly positive probability on every chance branch; and require that choices at each information event be generated by consequence-based rules that do not change when the same continuation problem is embedded in a larger tree. Adding continuity then yields an expected-utility representation at each event. The stronger conclusion (one state-independent utility, with subjective probabilities) needs a common consequence domain and further richness assumptions. §5.1 gives the full statement. Removing any of these assumptions can reopen a ✕ cell.
(b) The resolute-and-EU agreement holds under ordinary Bayesian conditioning, a stable utility function, chance that does not depend on which plan was chosen, feasible plans that can be recombined branch by branch, and a consistent rule for breaking ties. §5.2 explains why each is needed.
(c) The §4.2 calculation uses log utility of wealth. That is also the growth-rate rule only if the percentage gains and losses are understood as recurring in a multiplicative process; fixed dollar amounts repeated without scaling would define an additive process instead, where the growth-rate rule ranks by expected wealth.
So, behold! I think it is already quite a useful map, to be honest, even without the claims about the map itself. I think it is at least a useful thing to see what can be studied in decision theory. But I suggest going meta and studying the map itself — more on this is at the end of the post, after I build more examples and intuitions.
3.2 What each proposal actually specifies
The first table asked what happens when a scoring rule is combined with a decision setup. Its rows, though, are bundles: "causal supposition, re-solve at each decision point" fixes two things at once, because that is how the literature packages them. In the second table below, let's do the opposite. We take proposals one at a time and pull each apart into the pieces named in §2 (the rule that scores a prospect, the rule that turns a candidate into a prospect, and the rule relating earlier choices to later ones) so that one can see what each proposal actually settles and what it leaves open.
That last part is the point of the exercise. Many entries in the third column read "not settled by…": they mark places where a theory that gets discussed as though it were complete in fact leaves the question of later choices to be supplied from outside.
Scoring rule
How a candidate becomes a prospect
How earlier and later choices are related
Example or status
Expected utility
causal supposition
Not settled by the supposition rule. Standard practice is simply to solve the problem again at each decision point.
Classical CDT, in the versions due to Lewis, Skyrms and Joyce; their formulations differ in detail (overview)
Expected utility
evidential supposition
Not settled by the supposition rule. Same standard practice: solve again at each decision point.
ordinary acts and states, or a constructed logical prospect
Disputed; three variants are on offer. Re-solve as information arrives; anticipate your own future re-solving and plan around it (sophisticated); or carry out the plan chosen earlier (resolute).
Dynamic status disputed; §4 gives local FDT- and UDT-style hybrids, not a complete new theory
Global or nonseparable evaluation
ordinary chance
Resolute, or history-sensitive: what remains is valued partly by what has already happened, so the agent does not treat the continuation as a fresh problem (Machina).
McClennen- and Machina-style responses to preference reversal
Can violate independence, as his Alice-and-Bob example does; the updateless standpoint is part of the bundle, not a consequence of taking a geometric mean
compatible in principle with several supposition rules
Not settled by the scoring rule. Scoring by growth rate says nothing about whether an earlier plan binds a later choice.
Expected utility within each canonical fixed dynamic considered here; not one fixed-utility EU theory across the shared outcome domain of §8.1
Maxmin over several priors, applied recursively
ordinary acts and states
Recursive: today's value is defined through the values of the possible continuation problems, which is what makes the earlier plan and the later choice agree (Epstein–Schneider).
Obtained by restricting the prior sets (rectangularity) and updating them one prior at a time[19:1]
Maxmin over several priors
ordinary acts and states
Updating that depends on the choice problem: how beliefs are revised may depend on which options were on the table, which is how consistency is recovered (Hanany–Klibanoff).
Hanany and Klibanoff's dynamically consistent rules
Ambiguity-sensitive evaluation, general
supposition and updating rule left open
Sophisticated: anticipate what the future selves will choose and plan around them (Siniscalchi develops this under ambiguity).
Siniscalchi's analysis accommodates arbitrary decision models and updating rules
Expected-utility comparisons inside a stability test
causal or evidential, depending on the theory
Not settled by the test. Ratification applies within a single deliberation and leaves the question of later choices open.
Ratification is an extra condition on deliberation, not a rule about choices at different times[22]
Incomplete preferences, returning a set of permissible options
ordinary state-contingent plans
Global maximality: whole plans are compared, rather than choices being made step by step, which lets gaps between options survive (Petersen).
Petersen's framework is dynamically consistent while preserving those gaps
So, the prospect-evaluating rule has been studied quite a lot: before 1990, economics and philosophy already contained work by Allais, Ellsberg, Strotz, Machina, Quiggin, Schmeidler, and others, as well as prospect and disappointment theories. LessWrong has focused heavily on supposition rules while usually keeping EU fixed, but Garrabrant's work and infra-Bayesian work are important exceptions.
A rather side note: Buchak's position in the table is a useful example of why the labels about coherence and "valid cells" in the table must remain conditional. Her formal examples mostly use ordinary acts and states. The right dynamic interpretation of REU remains disputed. Thoma, Weatherson, and Pettigrew, Campbell-Moore & Konek (2025) identify pressures from choices over time and from choices about which risk attitude or decision rule to use.
4. Two illustrative calculations
This section is very easy and may be obvious for some, but I thought it may be helpful for others to demonstrate what I mean on a very concrete level. These are just numerical examples that separate the three proposed questions of decision theory by holding some parts fixed and changing others. Of course, each example is local: a numerical verdict in one problem doesn't allow per se to define a complete decision theory. Anyway, feel free to skip if obvious.
4.1 Newcomb's problem: the supposition rule does most of the work
Consider the standard Newcomb setup, in a probabilistic version. Omega predicts whether you will take only the opaque box or both boxes. The opaque box contains $1 million if Omega predicted one-boxing and nothing if Omega predicted two-boxing. The transparent box always contains $1,000. Omega's predictions are 99% accurate, which here means 99% conditional accuracy in either direction: given either contemplated output, Omega matches it with probability .
CDT and FDT can agree on all of those stated facts. In particular, they can agree that the opaque box was filled before the choice at the boxes and that physically taking the transparent box adds $1,000. Yet, they still construct different decision-relevant prospects. Standard act-level CDT causally sets the present box-taking act and does not let the earlier prediction or box contents vary with that intervention. FDT varies the decision function's output and, under the stipulated logical dependence, lets Omega's prediction vary with it. The disagreement is about the hypothetical comparison used in deliberation, not about whether the present physical movement reaches backward in time and changes an already filled box.
Suppose the agent has linear utility in money and uses Buchak's risk function . This function gives less weight to favorable outcomes when they are not certain, making the agent strongly risk-averse in the REU sense.
Standard decision-time CDT + REU chooses both boxes for every ordinary REU risk function. In the usual causal model, Omega's prediction and the contents of the opaque box are already fixed before the current choice. If we hold them fixed, taking both boxes gives $1,000 more in every state than taking only the opaque box. Standard REU respects this kind of state-by-state improvement, so the agent's risk attitude does not change the verdict.[23]
A note: let's not argue about whether CDT can be formulated in terms of policies. Merely writing the candidate as a policy does not, by itself, change the intervention point or the result. If CDT instead evaluates an earlier choice to install a one-boxing disposition before Omega predicts, that earlier choice can causally affect the prediction and CDT can endorse installing the disposition. Some authors use policy-CDT for a third construction: ex-ante optimization over whole policies even when no actual commitment was available. That construction can one-box, because the hypothetical policy choice is evaluated from a point causally upstream of the prediction; evaluating from earlier credences alone, with the intervention still at the boxes, would leave the dominance verdict unchanged. The later act-level problem is different, and an agent that optimizes again at the boxes may still two-box.[24]
A local FDT-style supposition + REU calculation chooses only the opaque box. Under the stipulated logical or subjunctive connection:
One-boxing gives $1,000,000 with probability 0.99 and $0 with probability 0.01:
Two-boxing gives $1,001,000 with probability 0.01 and $1,000 with probability 0.99:
So, the evaluation rule still matters: under the same stipulated prospects, a sufficiently pessimistic risk function would return to two-boxing whenever
The supposition rule does most of the work in this version of Newcomb's problem, but it does not do all of it.
4.2 Counterfactual mugging: evaluation matters even after the policy prospect is fixed
Now, Omega, assumed here to predict the relevant policy perfectly, flips a fair coin. On tails, Omega asks you for $100. On heads, Omega pays you $$Z$, but only if it predicted that you would have paid on tails. You wake up and see tails.
An ordinary act-by-act agent that conditions away the heads branch refuses to pay. An updateless formulation instead compares whole policies from before the coin result is observed.
From that earlier point of view, the “pay on tails” policy gives:
The “refuse on tails” policy gives $$0$.
For the first two calculations below, utility is linear in money; the REU calculation changes only the probability weighting. The third calculation instead uses log utility of final wealth.
UDT-style policy choice + linear EU: paying is strictly preferred exactly when
so the threshold is $Z>$100$; at $Z=$100$, paying and refusing tie.
UDT-style policy choice + REU with : the value of paying is
Paying is strictly preferred exactly when $Z>$300$; at $Z=$300$, paying and refusing tie.
UDT-style policy choice + log-wealth EU, starting with wealth $1,000: paying is strictly preferred exactly when
The exact threshold is $Z>$1000/9\approx $111.111\ldots$; at exactly $Z=$1000/9$, paying and refusing tie. If payments must be whole cents, the smallest strictly preferred amount is $$111.12$. If these percentage changes in wealth are understood as factors that recur in a stipulated multiplicative process, this is also the canonical multiplicative-EE calculation. Fixed dollar gains and losses repeated without scaling would instead define an additive process, so the EE interpretation requires this extra assumption about the environment.
Now set $Z=$250$. Linear EU and log-wealth EU pay. Under the multiplicative interpretation just stated, EE pays as well. The stated REU rule refuses. So, a policy-oriented supposition rule by itself does not settle the decision without an evaluation rule.
But this example distinguishes linear utility, logarithmic utility, and REU. It does not distinguish fixed-environment EU from EE. To make the latter distinction, we must compare different dynamics; §8.1 returns to this.
5. Compatibility results as an illustration of the entanglement of dimensions
This is probably the core of the article! Initially, I tried to "draw permanent walls between cells": what cells are prohibited, which are allowed, which are open in the map of decision theories. This... didn't work out well. What happened eventually was rather a collection of results with the form:
Given assumptions A–D, representation or behavior R is possible, required, or excluded.
Four groups of results come from the existing literature. The appendix also gives two AFAIK novel observations, labelled Theorems 7 and 8 there.
5.1 Hammond: demanding consistency across decision trees leads to expected utility
Suppose you insist that an agent's choices "hang together" across every decision tree it could face. Not merely that it ignores branches it can no longer reach — something considerably stronger: that whenever the same continuation problem comes up, whether standing alone or buried deep inside a larger tree, the agent handles it the same way. Hammond (1988) proved that an agent meeting that demand, across a rich enough collection of trees, must rank its options as though it were maximizing expected utility at each point where it has information.
That is a strong argument for expected utility, if you accept the demand. But there are two things it does not establish:
First, it is not a proof that any agent that updates its beliefs and chooses again must use expected utility. The requirement Hammond imposes is much stronger than "learn, then re-decide."
Second, what the argument delivers is local: an expected-utility representation at each information event, taken one at a time. The "conventional" EU theory (a single utility function, together with subjective probabilities, governing every event at once) needs further assumptions on top.
A theory that rejects expected utility can escape the conclusion by rejecting one of the premises, and several theories discussed here do exactly that. But it is not a free escape. A theory that denies a premise owes an account of why the premise was too strong and of what takes its place.
The precise statement, feel free to skip.Hammond (1988) fixes a nonempty finite state set and state-contingent consequence domains, then studies choices on the domain of all structurally admissible finite consequential decision trees built over that setup. Every chance-node transition must have strictly positive probability. His condition is stronger than the ordinary idea that an agent should ignore branches it can no longer reach. At each nonempty information event, choices must reveal a consequence rule that does not depend on how the same problem is drawn or placed inside a larger tree. Consequentialism together with consistency in subtrees yields a revealed ordering satisfying independence, as well as a restricted sure-thing principle for independent product distributions. Adding continuity gives a conditional expected-utility representation for each event.[25]
5.2 Rank-dependent rules can reverse later choices; resoluteness blocks one money pump
Why the reversal happens. A rank-dependent rule weights an outcome partly by where it ranks among the gamble's possible outcomes. The worst outcomes might count for extra, for instance. That ranking is a feature of the gamble as a whole. So when information arrives and some possibilities drop out, the survivors can move up or down the ranking and be given different weights. An agent that simply re-applies the same formula to whatever is left can therefore end up preferring, at the later point, the opposite of what it planned.[26]
Whether that counts as irrational doesn't seem to be a settled question, because "dynamic consistency" names several different demands. Must the later action match what the earlier plan specified? Must the later ranking of the remaining options match the earlier ranking of them? Or must the agent merely never end up worse off in every state than it could have been, in which case a reversal matters only if trades exist that exploit it? Theorists with global, nonseparable preferences reject the framing outright: what was assessed earlier is a whole gamble, part of whose value lay in branches that will now never occur, while what is assessed later is a fragment of it, so the two rankings were never required to agree. The reversal therefore settles less on its own than it appears to, at least in academic debates.
What is clear is that it opens a specific vulnerability. Gustafsson constructs a pair of trades that such an agent accepts one at a time, and that leave it worse off in every possible outcome than if it had refused both. Wakker exhibits a related cost: an agent whose preferences violate independence, and who has not committed to a plan, can prefer not to be given free information.
The precise premises of both results, feel free to skip. In §5.2 of Gustafsson's Independence Money Pump, the starting point is a particular strict independence reversal: , but mixing each with the same reverses the ranking. Backward induction, unidimensional continuity and decision-tree separability, the availability of the required offers, and two explicit protection principles then yield the pump.[27] Gustafsson's later treatments of stronger or biconditional formulations of independence require further premises.[28]Wakker (1988) also shows how an independence violation can produce a situation in which an uncommitted later chooser prefers not to receive free information, assuming complete, transitive, and suitably continuous preferences over his Jensen-style mixture setup and ceteris-paribus acquisition of the information.[29]
But expected utility avoids it. For an expected-utility agent, scoring a plan now and scoring it after the information arrives fit together automatically. The reason is that averages of averages are averages: the value a contingent plan has today equals the average of the values it will have once the agent knows more. So the plan that looks best in advance has continuations that look best later, which is exactly why the standard method of solving a decision tree backward works.
The precise formulation of why EU implies this kind of dynamic consistency, feel free to skip. EU has a clean way of composing earlier and later evaluation when probabilities are plan-independent, observation only reveals which information cell occurred, Bayesian conditioning is used, and feasible plans permit branchwise recombination. The last condition means that a feasible continuation at an information cell can be spliced into an otherwise feasible complete plan without violating a constraint linking different branches. The tower property then says that the earlier expected value of a contingent plan equals the expectation of its later conditional expected values.[30] On positive-probability information cells, a strictly better later continuation would therefore have produced a strictly better earlier plan after such a splice. This is why the standard EU method of solving that kind of decision tree backward works.
But there are different protections, available without expected utility! For example, Theorem 7 in the appendix offers a guarantee that does not require the agent to score gambles by expected utility. Informally: look at the whole tree in advance; score complete plans by any rule that always prefers a plan doing at least as well in every state and strictly better in at least one; pick a best plan and carry it out. Such an agent cannot be walked into a sure loss. In particular, if "refuse every offer" was among the plans it considered, no sequence of trades already present in the tree can leave it worse off in every state than refusing would have.
But now, look at what changes between those three findings! In §5.1 the scoring rule gets pushed toward expected utility, but the thing doing the pushing is a demand about decision trees, not a claim about how to score gambles. In the first half of §5.2 a rank-dependent rule walks into a money pump, but only once it is re-applied afresh at each new situation, which is again a fact about the dynamic setup rather than about rank-dependence itself. And then, Theorem 7 closes that vulnerability without touching the scoring rule at all: the same rank-dependent agent, told to fix a plan in advance and carry it out, cannot be led into a sure loss.
The moral is that in each case what decided whether the agent was in trouble was the pairing: how it scores gambles, together with how it relates earlier choices to later ones. That is the claim I state abstractly in the beginning, and here we have concrete theorem examples for it, exhibiting cases in which holding the scoring rule fixed and moving the dynamic setup flips the answer.
Note two consequences:
First, "is this decision theory coherent?" has no answer until a package is specified. When someone says a non-expected-utility rule is incoherent, the correct question is: incoherent paired with what? Paired with naive re-optimization, yes: that is Gustafsson's pump. Paired with ex-ante commitment in a foreseen finite tree, no: that is Theorem 7. Neither answer is a fact about rank-dependence alone.
Second, a theory that rejects expected utility incurs a debt it cannot pay on the evaluation axis. It has to say what happens when information arrives, and then accept whatever that answer costs: Hammond's premises rejected and something put in their place, or commitment together with the standing objection that it acts against what the agent, knowing more, would now prefer. In §7, we will collect those bills.
5.3 An interesting cross-term: when is it good to know things?
Consider an old question: should a rational agent ever pay to avoid seeing free evidence?
Good (1967) proves that free information has non-negative value for an expected-utility maximizer that conditions on what it observes, provided that:
the observation comes from a fixed list of mutually exclusive and jointly exhaustive alternatives, and tells the agent which one holds, so whatever it learns rules out everything it might have learned instead;
observing does not itself change any event that affects payoffs;
the acts available before observing remain available after every signal, so the agent can ignore the information in its action; and
the agent may choose a different action after different signals.[31]
Guess what I am going to say? Each part matters! The value of information therefore depends on like everything: the evaluation rule, the supposition and learning rules, the observation model, and the agent's ability to choose or commit to policies!
Remove any one of them, holding everything else fixed, and the guarantee can fail:
Change the evaluation rule.Wakker (1988)'s construction needs only an independence violation: in his Jensen-style setting of complete, transitive, and suitably continuous preferences over probabilistic mixtures, the agent still conditionalizes, still faces an ordinary partitional signal, and would still rather not look.
Change the supposition rule.Nielsen (2024) works in an expected-utility framework for “suppositional” decision theories and lets both the act-supposition operator and the learning rule vary. Under his assumptions the value-of-knowledge principle holds exactly when act supposition uses "Stalnakerian imaging" (a form of CDT; the details don't matter here) and learning uses conditionalization. Keep expected utility and change how a candidate act is supposed, and the guarantee goes.
Change the observation model.Ahmed & Salow (2019) have one example that keeps expected utility and conditionalization throughout; what breaks is the assumption that the signal partitions the possibilities. (Their other example instead uses Allais-style preferences that violate independence.)
Now notice that the dependence runs the other way as well! Theorem 8 in the appendix changes only the dynamic coordinate (the agent fixes a policy in advance rather than deciding after the signal arrives), and non-negative value comes back for evaluation rules that Wakker's argument would otherwise catch.
That pairing is the heart of it. The same evaluation rule is safe or unsafe depending on the dynamic setup. The same dynamic setup is safe or unsafe depending on the supposition rule. So "can free information hurt?" is not a question about any one of the three, and there is no way to answer it by reading off one coordinate at a time and combining the answers afterwards.
This is a nice illustration of the fact that the three questions are not separable in practice. If they were, one could settle the value of information by looking at the evaluation rule alone, and every result in this subsection would be about expected utility versus its rivals.
5.4 Intersections in Damascus
In Death in Damascus,[32] choosing either city can be unstable: settling on Damascus is evidence that Death predicted Damascus, while switching to Aleppo is evidence that Death predicted Aleppo. Under EU, and assuming that Death cannot predict the result of a private randomizer, a mixed strategy can sometimes be self-ratifying.[33] The ordinary proof uses linearity in mixtures: if the pure actions in the mixture have the same expected utility, then every mixture of them has that value.
So, while two previous subsections before this one paired the evaluation rule with the dynamic setup, this one exhibits a different pairing, and a more surprising one: a solution concept belonging to Question 2 (suppositions) turns out to rest on a property of the answer to Question 1 (evaluating prospects).
The standard treatment of Death in Damascus rescues the agent with a mixed strategy, and the rescue works because expected utility is linear in mixtures. This is not a fact about predictors, about ratification, or about causal reasoning, but a fact about how expected utility scores mixtures. A construction that looks purely dimension-2 is actually leaning on a dimension-1 property.
6. Description and identification
6.1 The same behavior can have several explanations, but different points in decision-theory-space are different
One way to summarize what we saw in §5:
Specific restrictions on one axis of decision theory may force some properties of other axes.
There are multiple ways of combining several axes to fix the same decision-theoretic problem.
But the second statement seems to be true only within certain limits! Observed choices do not uniquely reveal an agent's internal representation. On a small menu, we can often reproduce the same behavior by changing utility, beliefs, the supposition rule, or the dynamic solution, but there are limits to that. If we go "far enough" in decision-space, the solutions given by the different cells in the map of decision theories seem to eventually diverge.[34]
Also, some cells can explain almost any behavior if interpreted conveniently. For example, EU representation can almost always be restored by expanding the outcomes. For example, the consequence might include disappointment or the menu from which the outcome was chosen. But this changes the model.[35] If we are allowed to put arbitrary action or menu labels into the outcomes, many finite choice patterns can be made to look like EU, and the claim becomes weak and eventually unfalsifiable. A serious proposal to enrich outcomes should still have some restrictions on what counts as different options and use the same rule across all relevant problems.
6.2 Use panels of environments to identify the parts
Different changes to a decision problem test different parts of a theory.
What we want to test
Useful change to the problem
Warnings
How the agent scores a gamble. And, for growth-rate rules, whether it uses the same rule everywhere ()
Hold fixed what each option puts at stake and vary the probabilities: offer the same outcomes in different mixtures. For growth-rate rules, additionally offer the same wealth gambles once in an environment declared additive and once in one declared multiplicative.
Changing the dynamics changes the environment as well, so the cross-environment version asks whether one rule covers the whole family. It does not pin down which rule is at work inside any single environment.
What the agent supposes when it considers an action ()
Vary how the act is connected to the rest of the world: put in a common cause, a predictor, or a copy of the agent.
A predictor is not needed to separate CDT from EDT. They already come apart in plain common-cause cases such as the smoking lesion, so finding a difference does not show that a predictor was doing the work.
How earlier and later choices are related ()
Vary timing and information: when the choice is made, what has been observed by then, whether committing in advance is possible, and whether the later offer was foreseen.
Four postures are in play: naive, sophisticated, resolute, updateless. Any two of them can agree in a given tree while differing elsewhere.
How an observation the agent actually makes changes its beliefs ()
Hold fixed what the agent supposes when weighing a candidate, and vary only how a real observation revises the model.
These are two different operations, and the test only works if they are kept apart: conditioning on evidence that arrived is not the same thing as supposing an act for the sake of comparison (§2).
Whether the agent values randomizing for its own sake
Add or remove access to an external coin, leaving the pure actions exactly as they were.
The answer depends on how a predictor responds to the coin, so what the test measures is the agent-and-environment pair, not the agent alone (§5.4).
A single decision can sometimes distinguish theories if its menu is rich enough. Allais choices test independence; Newcomb-like choices can test supposition rules when the evaluator and background model are held fixed. What one observed choice usually cannot do is identify a single cell in the map. For that, we need a panel of problems and explicit limits on which alternative representations are allowed.
7. Dropping axioms one at a time: the islands of stability
There is a second, more radical way to generate the space of decision theories. Instead of collecting the theories people have actually proposed, we can start from expected utility's own axioms and ask, one at a time: what happens if I drop this one? The von Neumann–Morgenstern axioms together with the reduction-of-compound-lotteries axiom and its time-domain cousin, stationarity of discounting,[36] each generate a family of theories when relaxed.
Of course, there will be bundles violating any subset of the classical axioms you care to name. The classical literature's implicit position is that nearly all of it is uninhabitable: violate an axiom and you get money pumps, plans dominated by alternatives you could have taken, choices that reverse themselves as information arrives, equilibria that fail to exist. On that view the map needs one continent: expected utility with Bayesian updating. And a warning label for everything else.
But the warning label is too coarse, and I already showed four places where it misleads. This is for structural reasons, I think: every classical coherence argument is conditional. Each axiom is punished-when-violated only by a theorem whose premises include the other classical commitments. Hammond's derivation of independence consumes strong tree-consequentialism and re-optimization at each node (§5.1). Good's theorem on the value of information consumes expected utility, conditionalization, and the assumption that acts carry no information about states (§5.3). Gustafsson's money pump consumes re-evaluation at the later node (§5.2). So a single deviation from the classical bundle sits squarely in the punishment zone, which is what makes the warning label look justified. But correlated deviations can disable each other's punishers, because the punisher for one axiom is a theorem that needs the others.
Like in nuclear physics, there is a valley of stability: the expected-utility column, welcome in every row of §3.1's table. There are decay channels: money pumps, aversion to free information, equilibria that fail to exist, rules that undermine their own use. And there are islands of stability — bundles of mutually protective deviations, each certified against a named channel:
Resolute plus rank-dependent — the pump channel, closed against the route Theorem 7 covers. An agent that scores whole plans by a rule respecting state-by-state improvement, and that fixes a plan in advance for a foreseen finite tree and carries it out, cannot be walked into a sure loss. Toll: it must sometimes act against what it would now prefer, having learned more, which is the standing objection to resolute choice.
Commitment plus any menu-independent evaluation[37] — the information channel, closed by Theorem 8. If every act available without a signal survives as a policy that ignores the signal and yields the same prospect, then the offer of information only enlarges the menu, and an agent cannot be made worse off.
Maxmin over several priors, in two different repaired forms — the dynamic-inconsistency channel, closed twice over by two different correlated deviations. Epstein and Schneider impose dynamic consistency as an axiom and show it is characterized by "rectangular prior sets" together with prior-by-prior updating: the beliefs are restricted until no decision tree can expose the violation. Hanany and Klibanoff instead keep the beliefs and bend the updating, letting how beliefs are revised depend on the choice problem. Tolls: restricted beliefs in the first case and the abandonment of consequentialist updating in the second.
Global preferences, and incomplete ones — the more classical "islands of stability" of the economics and LessWrong literatures respectively. Machina-style non-separable preferences let the value of what remains depend on what has already happened, so the continuation is never treated as a fresh problem. Petersen's global-maximality framework drops completeness, so it allows pairs of options the agent neither ranks nor counts as equally good,[38] but then blocks the resulting exploitation argument by comparing whole plans instead of checking each step against what is locally available.
Two caveats, however:
First, there are islands and there are mirages. Some apparent violations are not new territory at all. Final wealth under compounding and the single-level geometric mean both look non-linear and both rank prospects exactly as an expected-utility rule does. Those are not islands but re-descriptions of the mainland, like the same rankings in different notation. An actual island is a bundle that no fixed re-description maps back to expected utility across all the environments in question.
Second, stability is always relative to a channel and always assumptions-based, never absolute. Theorem 7 certifies the resolute rank-dependent island against money pumps in finite, fully foreseen trees; remove foresight and the channel reopens. And a different channel stays open for the same island: Pettigrew, Campbell-Moore & Konek (2025) show that risk-weighted rules can be self-undermining when the agent must choose which decision rule to use, and nothing here closes that.
The same pattern appears on an axis I have not yet touched: time. Stationarity together with dynamic consistency lead to exponential discounting (Koopmans 1960; Strotz 1956). Exponential discounting is to the time axis roughly what expectation is to the risk axis: the form under which re-planning changes nothing.
Each axis has one identity of the same form: evaluating in two steps gives the same answer as evaluating in one. On the time axis that identity is ; on the risk axis it is the tower property. And on each axis one rule satisfies it: exponential discounting, uniquely so within the discounted-sum family, and on the risk axis expectation, whose conditional and unconditional versions compose in exactly this way.
The failure modes match as well! Every other discount function reshuffles the relative weights of future dates as they approach: the hyperbolic discounter is patient about distant trade-offs and impatient about imminent ones, which is Strotz's preference reversal. That is precisely what every non-expectation rule does to the relative weights of outcomes as information arrives.
And an agent whose rule fails the identity faces the same pair of options on either axis: accept that its later self will overturn its earlier plan, or bind itself in advance so that the later self cannot.
Ulysses took the second option. He knew that on hearing the sirens he would want to steer toward them, so he had himself lashed to the mast, not in order to change what his future self would want, but to ensure it could not act on the want. A hyperbolic discounter equipped with that kind of device is perfectly coherent, and behavioral economics rediscovered the arrangement under the name "commitment devices." In the vocabulary of §3.1, Ulysses belongs in the resolute row: his deviation is on the time axis rather than the risk axis, but the repair is the same one, and it works for the same reason.
My takeaway from this detour is that the "island principle" is not an artifact of risk, or of anything peculiar to lotteries. It is simply what conditional coherence theorems do. Each one names a set of premises, and each can be escaped by giving up a premise and paying for it elsewhere. On whatever axis such a theorem is stated, the same situation appears.
8. Toward a metatheory of decision theories
The three-question framework suggests a research program — call it a metatheory of decision theories: its objects are whole packages (evaluation rule, supposition rule, dynamic solution, learning rule), and its results are the conditional compatibility theorems that link them.
In particular, for any proposed decision theory, record:
its feasible candidate space , including history-dependent menus, together with its prospect space and supposition rule ;
its evaluation rule , including exactly what counts as an outcome;
its structured dynamic solution , including selection standpoint and execution or re-optimization;
its learning rule , unless that rule is explicitly fixed as part of the background model;
the family of environments over which the theory is meant to apply; and
each relevant coherence result, together with all of its premises.
At the very least, this bookkeeping blocks several common category errors:
A theorem about strong tree-consequentialism does not automatically apply to every form of updating.
A commitment result for a finite foreseen tree does not settle cases with unexpected offers or logical predictors.
An EU representation inside each environment does not show that one utility function works across changing dynamics.
Matching a finite set of observed choices does not identify one unique internal representation.
But there may be more to it!
8.1 A gauge analogy: descriptions and invariants
There is a useful analogy here with gauge redundancy in physics and, even more directly, with Klein's Erlangen program in geometry.[39] The basic idea is that several mathematical descriptions can represent the same observable object, so we should ask which features change with the description and which features remain invariant.
Part of that idea can already be stated exactly. Fix:
a family of environments ;
one shared language for actions and outcomes; and
a rule for which redescriptions are allowed.
Let be a complete theory schema over . For each environment model , the schema supplies a type-compatible local specification , together with a separate learning rule and any other supplements when they are not already fixed by . Let be the agent's complete pattern of choices across the environments in . We can define
Thus and are behaviorally equivalent on the stated environment family. The particular formulas used in or are description-dependent. Their shared behavior is invariant under any transformation that takes one description to the other while preserving that behavior.
I find that this makes the use of the geometric language surprisingly efficient!
Term
Defensible meaning here
Current status
Coordinates
the five slots of the local specification , plus supplementary ones sometimes, relative to a fixed model and outcome language
useful formal bookkeeping, but can't call it a "coordinate system" in the strict sense yet, as objects of different categories can still sneak into some coordinate
Invariant
the complete choice pattern , or any property determined by it
exact once and the admissible descriptions are fixed
Gauge-equivalent descriptions
two descriptions connected by a defined, reversible redescription that preserves
exact for particular transformations; not yet one general theory
Equivalence class
all descriptions behaviorally equivalent under
exact
Orbit
the descriptions reached from one description by a specified family of reversible transformations
only an analogy yet, until an action of the transformations on the descriptions is defined
Chart
a proposed numerical parametrization of a region of descriptions
only an analogy until a space, covered regions, and compatible changes of coordinates are defined;
There are already several full-fledged examples of description-invariance, old and new:
Replacing a von Neumann–Morgenstern utility by , with , changes the numbers but not the lottery ranking.
A single-level geometric expectation and expected log utility display different formulas but rank the same positive prospects identically.
In one fixed multiplicative environment, the usual EE evaluator and EU with log-wealth utility rank the same induced prospects identically. They give the same complete choices when the feasible policies, supposition and learning rules, dynamic solution, and treatment of ties are also fixed.
The EE example also shows why the environment family must be named. In each fixed additive or multiplicative environment, the EE rule has an EU representation. Across both environments on one common wealth domain,[40] no single fixed utility represents the whole pattern.[41] “EU-representable” is therefore true locally and false uniformly. That exact failure can motivate the picture of local descriptions that do not combine into one global description.
Still, the "gauge representation" may be a real thing but may also be just an analogy. But the analogy is still useful because it changes the questions we ask:
Over which environment family are two descriptions equivalent?
Which transformations provably preserve behavior on that family?
Which features remain invariant under those transformations?
Which additional environments separate descriptions that looked equivalent on a smaller test set?
Does an equivalence class contain one fixed-EU representation, only environment-specific EU representations, or no EU representation on the stated domain?
This is close in spirit to the Erlangen program: study a space through its transformations and invariants.
My previous post argued that the independence axiom is not the boundary of rationality. This post has tried to say what the territory beyond the boundary is actually shaped like: not a free product of independent choices, but a constrained space whose constraints are theorems. The zoo has a geography: a mainland, a sea whose currents are the coherence theorems, and islands of correlated deviations that certify each other's safety. I have tried to draw the first map coarse enough to be checked and corrected.
And if the zoo has a geography, the natural next question is what the terrain is made of — what the space of decision theories actually is, mathematically. There are hints scattered through this post that the answer is algebraic. Klein's Erlangen program characterized each geometry by its transformation group and its invariants. Whether the metatheory of decision theories admits an Erlangen program of its own, whether the zoo has not only a geography but an algebra, is an interesting question to ask.
The independence axiom says that if an agent weakly prefers lottery to lottery , then mixing each with the same third lottery , in the same proportions, should preserve that preference. In symbols, for , if and only if . The axiom is about preferences on a fixed lottery domain with a fixed meaning of probabilistic mixture. ↩︎
A prospect is the uncertainty-bearing description of payoff-relevant consequences that a theory uses to evaluate a candidate act or policy. In the simplest and most common case, it is one probability distribution: “$100 with probability , otherwise $0.” A multiple-prior theory may instead use a set of distributions, and a two-level theory may preserve more than one kind of uncertainty. Importantly, the prospect is not a property of the bare physical act alone, and it is not simply a prediction of what will actually occur. It is generated from the candidate, a background model, and a rule for supposing that the candidate is chosen. ↩︎
Utility is a numerical representation of preferences; it need not mean pleasure or money. Expected utility (EU) evaluates a lottery as , the probability-weighted average of the utilities of its possible outcomes. A positive affine change , with , leaves every lottery ranking unchanged, so the unit and zero point are arbitrary. An arbitrary increasing transformation need not preserve rankings under risk: the relative sizes of utility differences matter. ↩︎
A weak ordering is a ranking that is complete and transitive: every pair can be compared, and if and , then . Continuity or Archimedean regularity rules out certain infinitely sharp jumps in preference and ensures that intermediate probabilistic mixtures can represent intermediate rankings. The von Neumann–Morgenstern representation theorem uses these conditions together with independence. ↩︎
Rank-dependent utility (RDU) changes the decision weights according to an outcome's rank in the gamble, rather than multiplying each outcome's utility by its probability directly. Risk-weighted expected utility (REU) is Buchak's formulation within the same broad rank-dependent family, using a risk function . Ambiguity means uncertainty about the relevant probabilities, as opposed to risk under agreed precise probabilities. Multiple-prior models represent ambiguity with a set of first-order probability distributions; other models exist. For example, smooth ambiguity uses a probability distribution over candidate first-order probability models together with a separate attitude toward that uncertainty. “Two-level” here means that the model treats two sources of uncertainty separately—for example, ordinary chance within each possible world and uncertainty over which world or copy is yours. ↩︎
A supposition rule defines a controlled hypothetical comparison. It identifies what is being supposed—for example, learning an act, causally setting it, or varying a computation's output—and determines what remains fixed and what changes with that supposition. A counterfactual asks what would be true if something were different. A subjunctive dependence is a broader “would-change-if” connection. It may be causal, but in FDT it can also be logical: varying the output of a computation can vary the modeled outputs of predictors or identical copies, even when the computation does not physically cause their earlier behavior. ↩︎
Conditioning replaces a probability distribution with the distribution conditional on an event, such as “I choose .” It can change beliefs about causes and other facts statistically associated with . A causal intervention represents setting a variable to a value while preserving the rest of a causal model according to its equations; it changes downstream effects but does not treat the intervention as evidence about its ordinary causes. Logical or subjunctive approaches instead ask which represented facts would vary with the decision procedure's output. Even when all sides accept the same background causal and statistical model, these three operations can assign different decision-relevant prospects to the same visible act. ↩︎
A common cause influences both an action and an outcome without the action causing that outcome. In a simple smoking-lesion model, a lesion causes both smoking and cancer, while smoking itself is stipulated not to cause cancer. Whether EDT treats the contemplated act as additional evidence depends on what the agent already knows. In “tickle defence” versions, present evidence about the internal urge or decision process can screen off the act from the lesion, so EDT need not avoid smoking. The main text uses a version without that screening-off assumption solely to illustrate the difference between conditioning and intervention. ↩︎
Ex ante means evaluated from an earlier point of view, before a specified observation or event; ex post means evaluated afterward. An agent may update its beliefs after observing without abandoning a previously chosen plan, so “updates beliefs” and “optimizes again” must not be treated as synonyms. ↩︎
A representation result shows that a pattern of preferences or choices can be described by a particular mathematical form, such as expected utility. Dynamic consistency concerns agreement between plans made earlier and choices preferred later, once the relevant conditioning is specified. An equilibrium is a stable set of choices in which no participant—or no relevant future self—has an incentive to change unilaterally. Coherence is broader and potentially ambiguous; throughout the article, it should be read only through a named property or theorem. ↩︎
The Single Choice Principle (SCP) says, roughly, that making a conditional plan for one choice in advance and making that same choice when the condition is reached should give the same verdict. Its motivation is a Ramsey-style test: to ask what you should do if is true, imagine and ask what you should then do. Weatherson applies the principle by placing a chance event or predictor between the planning stage and the one actual choice. Buchak-style risk weights can evaluate the earlier compound prospect differently from the later reduced prospect unless the risk function is linear. The principle therefore pressures REU toward ordinary EU. A resolute or nonseparable theorist can reply that the earlier and later questions are not interchangeable, but then owes an account of why the disagreement is rationally acceptable. Weatherson's argument is an alleged inconsistency argument, not a money-pump argument. ↩︎
Ratificationism adds a stability test: an option is rationally available only if, after supposing that the agent will choose it, no other option becomes better by the theory's own standard. In predictor cases, a contemplated choice can change the evidence about what was predicted, so simple maximization can oscillate. A ratifiable choice does not undermine itself in this way. Causal ratificationism evaluates the candidate options by their causal consequences while applying this self-endorsement test. Depending on the model, more than one choice or a randomized choice can be ratifiable. ↩︎
A preference is additively separable across components if some numerical representation of its ranking has the form ; the definition allows strictly increasing changes of numerical scale. It is nonseparable when no such representation exists. A product formula is therefore not enough to establish nonseparability: for positive , has the separable representation . The two-level geometric example instead violates expected-utility independence under ordinary mixing of lotteries. ↩︎
A policy is a complete contingent plan. A pure policy is a function from possible observation histories to actions; it may say “choose after signal , but choose after signal .” A behavioral randomized policy assigns an action distribution separately at each history. A mixed policy draws one pure policy from an earlier lottery, which can correlate the actions prescribed at different histories. These representations are equivalent only under additional conditions. The model must also say when random bits are generated, whether a seed is shared across histories, and what a predictor can observe. Calling an object a policy says how it is represented, not when it is chosen, installed, or made visible to a predictor. ↩︎
A representation theorem states conditions under which some abstract preferences or choices can be represented in a specified mathematical form. It does not automatically provide the correct causal model, solve zero-probability conditioning, or explain how the representation should be used at later decision points. ↩︎
A Cartesian product is the set of all combinations formed by choosing one item from each of several sets. Here it would mean every possible triple of one evaluation rule, one supposition rule, and one dynamic solution. The article does not claim that every formally writable triple is a complete or coherent theory. ↩︎
Ergodicity economics (EE) connects decisions to the long-run growth rate of a quantity such as wealth under repeated dynamics; Peters and Adamou's time interpretation of expected utility explicitly presents the correspondence between dynamics-specific growth optimization and expected utility. In the canonical cases used here, stationary ergodic additive increments with suitable finite averages lead to maximizing expected wealth change, while positive stationary ergodic multiplicative factors with finite expected log lead to maximizing expected log-wealth change. For a specified process and measured quantity, ergodicity means, roughly, that the long-run average along almost every path agrees with the corresponding average across the probability distribution, when the relevant averages and limits exist. The article uses only these idealized additive and multiplicative prescriptions; it does not assume that every claim made under the EE label follows from them. ↩︎
An ordinary state-contingent act is represented as a mapping from possible states to consequences. The ambiguity papers placed in this row study preferences over such acts and learning rules for updating them. That framework does not, by itself, say whether Newcomb-like or common-cause cases should use causal intervention, evidential conditioning on the act, or a logical/subjunctive comparison. This is why the row is an updating package rather than a clean value of . ↩︎
A multiple-prior model represents ambiguity with a set of probability distributions rather than one distribution. Rectangularity is a structural condition that lets the set of priors be broken into compatible marginal and conditional pieces across information stages. A recursive rule defines today's value partly through the values assigned to possible continuation problems. These details are what make the relevant consistency result work; “uses several priors” by itself is not enough. ↩︎↩︎
The Kelly criterion chooses bets to maximize expected logarithmic growth of capital in a repeated multiplicative setting. A maxmin expected-utility rule represents ambiguity with several candidate probability distributions, calculates expected utility under each, and judges an act by the lowest of those values. Unrestricted maxmin updating can disagree with its earlier plan; special structures or updating rules are used to recover particular forms of dynamic consistency. ↩︎
Infra-Bayesianism is a family of attempts to generalize ordinary Bayesian models so that an agent can reason about richer uncertainty and about environments that may respond to its policy. It often uses sets of probability-like models or generalized expectation objects rather than one fixed probability distribution. The exact formalism varies. Calling a proposal policy-based does not by itself show that it uses FDT-style logical supposition or an updateless dynamic solution, so the table records only a related family rather than placing every infra-Bayesian proposal in the row. ↩︎
A deliberational decision theory does not simply maximize once under a fixed supposition about its own action. It looks for a recommendation that remains stable after accounting for what the agent's contemplated choice says about the decision problem. Ratification theories are important examples, although the exact use of the label varies across authors. This is a within-deliberation condition, not by itself an answer about how choices at different histories should relate. ↩︎
Statewise dominance means that one act gives at least as good an outcome in every possible state and a strictly better outcome in at least one state that has positive probability. A rule that respects it never chooses the dominated act solely because of its attitude to risk. In standard act-level Newcomb, two-boxing gives exactly $1,000 more for each fixed content of the opaque box. ↩︎
Writing candidates as policies is not the same as ex-ante policy optimization. Three constructions should be separated. First, Everitt, Leike, and Hutter define a sequential policy-causal intervention and prove, in their model, that it is equivalent to intervening on the current action; this version still gives standard CDT's two-boxing verdict. Second, an actual earlier commitment or disposition choice made before Omega predicts can causally affect the prediction, so CDT may favor installing a one-boxing disposition even though a later uncommitted CDT agent two-boxes. Third, Oesterheld and Conitzer use policy-CDT for following the whole policy that CDT would have preferred to commit to ex ante, even when no actual commitment was available; their ex-ante-commitment interpretation one-boxes in their Newcomb variant, where an earlier decision point precedes the predictor's observation — a dependence they note explicitly. Thus the familiar objection that policy-based CDT one-boxes is correct if “policy-based CDT” means this third proposal, but not if it means that the candidates in standard decision-time CDT have merely been written as policies. The article uses the mere-policy-representation setup unless it explicitly says “ex-ante policy optimization.” ↩︎
Tree-consequentialism, in the strong sense used here, requires choices in a decision tree to be generated by consequence-based choice rules that remain invariant when the same continuation problem is represented in different larger trees. Hammond first fixes a nonempty finite state set and state-contingent consequence domains, then permits every structurally admissible finite consequential tree over that setup while requiring strictly positive probabilities at every chance node. On that domain, consequentialism and consistency in subtrees imply independence for the revealed ordering at each nonempty event and a restricted sure-thing principle for independent product distributions. Continuity yields a conditional expected-utility representation at each event; this result does not require the later three-indifference-class assumption. To obtain the stronger additive, state-independent SEU representation and Bayesian relations, Hammond additionally uses a common consequence domain, state-independent treatment of constant consequences, at least three distinct indifference classes, and his other maintained assumptions. The resulting subjective probabilities are strictly positive, so null states are excluded. ↩︎
A conditional preference reversal occurs when an earlier plan ranks one continuation above another, but after learning which branch was reached the agent ranks the continuations in the opposite order. Such a reversal is not automatically irrational: whether it counts as dynamic inconsistency depends on which earlier and later objects the theory says should agree. ↩︎
A money pump is a sequence of individually accepted trades that predictably leaves the agent worse off, often with a sure loss. Money-pump arguments are conditional: they require a specified sequence of offers, an account of what the agent knows, and assumptions about whether it can commit or refuse. ↩︎
Gustafsson's §5.2 starts from the weak strict-preference failure of independence and , where is the same probabilistic mixture operation on each side. Unidimensional Continuity of Preference supplies a slightly worsened option needed in the construction. Decision-tree separability lets choices in a subtree be assessed without irrelevant parts of the larger tree. The Principle of Unexploitability says, roughly, that an ideal chooser does not knowingly select a foreseeably exploitable strategy. Preferential Invulnerability is a bridge principle: if a combination of preferences makes an agent vulnerable to violating a requirement of rationality in some possible situation, that preference combination is rationally prohibited generally. Gustafsson also assumes backward induction and that the required choices and offers are possible. His later arguments for stronger and biconditional versions of independence add further premises, so the §5.2 package should not be read as proving every formulation from any generic “independence violation.” ↩︎
A preference is complete if every pair of options can be compared and transitive if and imply . A mixture is a lottery that first uses an external randomizer to select with probability and otherwise. Wakker uses Jensen's continuity condition, which says roughly that sufficiently small probabilistic perturbations cannot reverse a strict preference. ↩︎
The tower property says that taking a conditional expectation and then averaging over the possible information states gives the original unconditional expectation: . It is sometimes called the law of iterated expectations. To infer dynamic agreement, one also needs branchwise recombination of feasible plans: replacing a plan's continuation on one information cell with another feasible continuation must leave a feasible complete plan. Otherwise a later action may be locally feasible even though it could not have been combined with the plan's other branches ex ante. The tower property also constrains continuations only almost surely; a separate updating or choice rule is needed at null histories. ↩︎
A partitional observation tells the agent which member of a mutually exclusive and exhaustive partition contains the true state. An inexact or non-partitional observation need not have this clean form. Good's result also assumes that observing does not change the payoff-relevant world, apart from changing what the agent knows and which contingent action it selects, and that the original act menu remains available after every signal. ↩︎
In Death in Damascus, Death predicts where the agent will go and waits there. If the agent settles on Damascus, that is evidence Death predicted Damascus; if the agent switches to Aleppo, the same reasoning points to Aleppo. The case is used to study unstable deliberation, ratification, and whether private randomization can help when the predictor cannot foresee the randomizer's result. ↩︎
A mixed strategy deliberately randomizes among actions. A choice is self-ratifying if supposing that it will be chosen does not make another choice better. A Nash equilibrium is a profile of strategies in which no player benefits by changing strategy alone, given the others' strategies. Standard existence results normally assume an expected-utility treatment of mixtures or related conditions that make preferences over mixtures sufficiently regular. ↩︎
But I am not sure. Maybe not, and there are actual full "invariants". See however section 8. ↩︎
Outcome enrichment means redefining consequences to include features that were previously left out, such as regret, disappointment, the menu, or the action label. This can make a formerly non-EU pattern representable as EU, but only by changing what the theory treats as an outcome. ↩︎
A time-separable discounted-utility model adds independently valued rewards from different dates after multiplying each by a discount factor. Stationarity says that preferences between time patterns do not depend on a common shift in calendar time. Under familiar extra assumptions, and after choosing the normalization , stationarity and dynamic consistency produce exponential discounting. Hyperbolic discounting declines roughly like a reciprocal function and can produce preference reversals; commitment can manage those reversals, which is the limited analogy to resolute choice. ↩︎
A menu-independent evaluator assigns a value to each available policy's prospect without changing that prospect's value when other policies are added or removed. Prospect extensionality means that two policies producing the same decision-relevant prospect receive the same value. A constant policy ignores the new signal and chooses the same old act after every signal. If every old act has an outcome-equivalent constant policy, the new menu contains a copy of the old menu. ↩︎
Incomplete preferences allow some pairs of options to remain incomparable. They cannot in general be represented by one real-valued score, because a single real number for each option automatically orders every pair, allowing ties. ↩︎
Klein's Erlangen program classified geometries by the transformations under which their central properties remain invariant: Euclidean geometry studies what rigid motions preserve, for example. In physics, a gauge redundancy occurs when different mathematical descriptions represent the same observable physical state. An orbit is the set reached by applying a specified family of composable transformations to one object. A chart is a coordinate description of part of a space with compatible overlap rules, and curvature requires additional geometric structure for comparing directions at different points. This article defines a behavioral equivalence relation and identifies some exact behavior-preserving redescriptions. It does not yet supply the general transformation system or geometric structure required to use all of these terms literally. ↩︎
A common outcome domain means that the same set of consequences is used across the compared environments. If the environment label is added to each consequence, “£1,000 in environment ” and “£1,000 in environment ” become different outcomes, which permits utilities that were impossible on the unlabelled domain. ↩︎
The von Neumann–Morgenstern uniqueness result says that two expected-utility functions represent the same non-trivial preference over a sufficiently rich lottery domain only if one is a positive affine transformation of the other: with . A Bernoulli utility function is the utility assigned to individual outcomes inside the expectation. An interval of positive length contains a continuum of possible wealth values, not merely two isolated points. ↩︎
TL;DR. LessWrong's decision-theory debates (Newcomb, FDT vs CDT, counterfactual muggings) are almost entirely about what we suppose when we consider a candidate action or policy. There is a second, older, semi-orthogonal, but not fully orthogonal question: how to score a gamble once you know the possible outcomes. The predominant (and mostly implicit) answer to that was "take the expected utility". This post treats the two questions, plus a question about how a choice made before receiving information should relate to choices made afterward, as separate axes, and maps every decision theory you have heard of (and several nobody has built) into the resulting grid. Interestingly, the axes are provably entangled: theorems old and new show that there exist different restrictions on what places in that "decision-theoretic space" are inhabitable. I think there is a structure, maybe a deep and consequential structure, inside this map of decision theories which shows what possible combinations across the axes are coherent and fruitful. If we study it, we may understand the entire set of all possible coherent decision theories, something akin to the "metatheory of decision theories". It may be useful to know the entire set.
This is both a self-educational note and a research post. I have tried to state the scope of every result. While I am not certain about every conclusion about the "space of decision theories," I am confident that thinking in terms of the proposed space is useful. In the most conservative case, it is a nice map of decision theories. In a more optimistic case, the space has non-trivial structure whose study may lead to new results. Note also that this is a rather technical post. I tried to make it as simple as I could, but at some point, you lose precision, and I didn't want to go further that route, because it is meant not only as an educational post, but also as a research one. That said, I think with some additional effort it may become more clear. In particular, I added many footnotes that clarify terms (without putting a lot of theory in the body itself), and I think more footnotes can be added. Most of the footnotes are AI-generated, prompted by me to focus on addressing some common cases of confusion. Please let me know if something should be clarified. Also, as the post sometimes goes quite technical and is rather long, feel free to skip some pieces which are boring to you. I'd rather have you read intro and final sections (7 and 8) than read only the first half of the post.
Technical appendix — formal statements, proof sketches, finite numerical examples, and open problems (for now not verified, many things AI-generated, comes with no guarantee!): appendix link.
1. Decision theory asks at least three questions
This post began as a sequel to On The Independence Axiom. That post argued that the independence axiom is stronger than rational choice requires. [1] Independence is enough, as part of a larger set of assumptions, to support expected utility. But I argued that rejecting independence does not by itself entail inconsistency.
The discussions that followed, both online and offline, taught me something about how LessWrong talks about decision theory. When I asked how we should evaluate a probability distribution over outcomes, people often answered with ideas about which distribution should be used or from which informational point a policy should be chosen: FDT, UDT, updatelessness, and counterfactual mugging. This is not bad, per se. Some of the responses were especially helpful, including Wei Dai's reference to his 2009 post; so was Scott Garrabrant's closely related point about updating, made in another discussion. But the responses were often answering a different question from the one I had asked.
I therefore wanted to clarify the separation of three questions.
Question 1: How should we evaluate a prospect?
Suppose each available action has already been matched with a prospect [2] —in the simplest case, a probability distribution over outcomes. How should we rank those prospects?
Expected utility theory says: give each outcome a utility, multiply each utility by its probability, and add the results. [3] This rule follows from the standard von Neumann–Morgenstern assumptions taken together: a weak ordering of options, suitable regularity or continuity, and independence. It does not follow from independence alone. [4]
Other answers are possible. Rank-dependent utility changes the weights placed on cumulative probabilities. Buchak's risk-weighted expected utility gives a closely related formulation. Garrabrant's two-level geometric rationality and rules designed to handle ambiguity are further examples. [5]
Question 2: Which prospect should be used for each candidate?
At the first rough glance, it is akin to the question of “what would happen if I chose this action?”, but this is too ambiguous. It can mean at least three things:
These are not simply three forecasts about what physically happens after the agent moves. Two theories can agree on the full causal and statistical model, including which physical effects each act causes, and still use different probability distributions in deliberation. They differ over which hypothetical comparison is relevant to choice. Advocates of the theories may also disagree about the background model, but that is a separate disagreement (for example, a disagreement may be whether the predictor is running a physical simulation of you or is instead a crude statistician relying on a correlation); the supposition-rule distinction remains even when the model is held fixed.
I will call the rule for making that comparison the supposition rule. [6] Given a background model and a candidate action or policy, it says:
The uncertainty representation that results from this operation (usually a probability distribution) is the candidate's decision-relevant prospect. A prospect is therefore not attached to the bare physical act alone, but is produced by applying a supposition rule to that act or policy within a model.
With that in mind, the main theories can now be stated more precisely:
For example, consider a version of smoking lesion in which a lesion causes both the choice to smoke and cancer, smoking itself does not cause cancer, and the agent's present evidence does not already screen off the correlation between its act and the lesion. [8] EDT then treats smoking as evidence of the lesion and evaluates it using the corresponding conditional distribution. CDT intervenes on smoking, leaves the probability of the earlier lesion unchanged, and therefore uses a different prospect. The disagreement is not over whether smoking physically causes cancer, but over which distribution is relevant when comparing the acts.
Question 3: How should choices at different times fit together?
An agent may choose before learning something, and then choose again after learning it. How should these choices be related?
Resolute and updateless agents may behave alike in some decision trees, but the ideas are not the same. Resoluteness describes an agent's relation to an earlier plan. Updatelessness describes the point of view from which the policy-selection problem is set up.
The standard von Neumann–Morgenstern representation of preferences over lotteries does not by itself answer Questions 2 or 3. It takes lotteries as the starting objects and does not specify how an act generates a lottery or how later choices should relate to an earlier plan. And much LessWrong work holds the answer to Question 1 fixed at expected utility and changes the answer to Question 2, Question 3, or both. The FDT paper says this directly:
The same paper approvingly quotes James Joyce's 1999 claim that rival decision theories need not offer rival theories of value; they may instead disagree about the point of view from which actions should be evaluated. UDT likewise used expected utility over logical or mathematical uncertainty as a placeholder rather than deriving it. That leaves open the question of how other evaluation rules combine with different supposition rules and different ways of handling later choices.
The division itself is not historically new. Joyce made part of it explicit, the FDT paper quotes him, and Lara Buchak's survey of decision theory covers several of the same distinctions. Still, it is worth making the division more visible. I repeatedly encountered confusion that would have been easier to avoid if the three questions had been stated separately.
However, this post is not about the division itself.
What I want to do here is to cross all these dimensions, and map the entire zoo of decision theories into the resulting space: occupied cells, promising cells, open cells, and cells that are provably closed. Or, to be more precise, to place familiar theories in this three-part space and state the known compatibility results together with their assumptions.
Here is the central claim:
Two immediate clarifying notes, however:
First, whether a combination is acceptable is not a property of the combination by itself. It depends on which demands you place on the agent and which environments you expect it to face. So the useful output I am trying to achieve is not a verdict stamped on each combination, but a record of the form: this bundle of answers, in this class of environments, satisfies this demand, under these assumptions, at this price. Some prices are substantial, like committing in advance to a plan you may later wish to revise, restricting which beliefs you are allowed to hold, or changing how you update, and naming the price is informative by itself. Note that initially I tried to find rather "universally consistent cells", but that simply didn't work. Maybe it's just me failing. But for all I see, all consistency results are very conditional.
Second, the failures are not scattered at random. They cluster around a small number of structural features that recur from theorem to theorem: how the agent treats branches it did not end up on, whether it re-solves the problem from scratch after learning something, and whether the environment responds to its whole plan rather than only to its actions. That recurrence is the reason I suspect this space has real interesting structure, rather than being a list of unrelated special cases.
So, the idea is that "is this way of scoring gambles coherent?" is not, by itself, a well-posed question. Coherence belongs to the whole bundle: how you score gambles, what you suppose when you weigh a candidate action, and how today's plan constrains tomorrow's choice. Demand a strong enough form of consistency across decision trees, and you are pushed to an expected-utility representation at each information event. Keep a rank-dependent rule instead and re-apply it as information arrives, and your later self can overturn the plan your earlier self preferred. Take one of the standard repairs (commit in advance to the plan, or restrict your beliefs so the conflict cannot arise), and you have bought protection against one named failure, on that repair's own terms and no further. Rejecting expected utility is therefore not automatically incoherent, but it is a commitment about the whole bundle.
Brian Weatherson's Four Problems in Decision Theory and Game Theory as Decision Theory are the closest neighbors to this post. His derivations from the Single Choice Principle (SCP) [11] are strong examples of the kind of compatibility result between the three axes I have in mind. For example, he argues that SCP forces Buchak's risk function to be the ordinary linear one, and that SCP plus further assumptions leads to a form of causal ratificationism. [12] This entire line of reasoning anticipates what I am trying to say, but more generally.
I do have some disagreements, but feel free to skip this paragraph as it is not super essential for the post. However, if you want to hear: In problems where the agent faces only one real choice, SCP demands that planning and doing agree: if it is rational to decide in advance "should I reach this point, do X," then on reaching that point, X must be what it is rational to do. The motivation is a Ramsey-style test, which treats two questions as one — "what should I do if p turns out to be true?" and "p is true; what should I do?" That identification is precisely what resolute theorists, and theorists with global, nonseparable preferences, [13] deny, and I think for a good reason: on their view the earlier question concerns a whole gamble, part of whose value lies in branches that will never be reached, while the later question concerns only a fragment of it. There are two different objects, so agreement between the answers is not automatic. SCP is therefore a compatibility result of the same conditional kind as those which will be described in §5, and Weatherson himself notes that violating it produces no money pump or sure loss, only an alleged inconsistency.
2. A compact formal picture (almost)
Write
In words, it is something like this: out of the plans available to you, pick the one whose associated prospect scores highest. Before that instruction means anything, three things must be supplied: the available plans, the rule turning a plan into a prospect, and the rule scoring prospects (and a fourth is needed as soon as the problem unfolds over time). The list of available plans
A theory that runs over time also has to say how an observation the agent actually makes changes its beliefs or its model. Call that its learning rule
Some theories need more structure still. For example, a ratification rule asks whether a contemplated choice still looks acceptable once the agent supposes it is about to make it. It is like a stability test applied inside a single deliberation, which is not automatically a rule about how choices at different times should relate. A theory with incomplete preferences may return a set of permissible options rather than a single best one. A menu-dependent theory cannot score an option without knowing what else is on offer, which breaks the assumption this section opened with. Where such features matter, the core list
Maybe, even having covered all that, I am still missing something, but to me it looks like a complete characterization of any decision theory. Still, we will focus only on the three questions stated at the beginning of this post.
3. The map of decision theories
The entire framework, as we saw, has three dimensions, so one flat table cannot display all three independently. The most useful two-dimensional view, as I think about it, puts evaluation and uncertainty families in the columns and decision setups, dynamic packages, or cross-tree constraints in the rows. This preserves comparison by dimension, but it is not a Cartesian grid [16] whose rows and columns are all values of one mathematical type. The following section then lists the components of partially and fully specified families more explicitly.
3.1 Two-dimensional classification
The table below crosses two main things I am trying to compare. Each column is a family of scoring rules: an answer to Question 1. Each row is a decision setup: a way of supposing a candidate action, together with a way of relating earlier and later choices: Questions 2 and 3 packaged as they actually usually appear in the literature. Each cell says what is known about putting that scoring rule together with that setup — a combination that amounts to a concrete decision theory.
I assign the cells one of the five statuses:
The columns.
A note on the rows. They are packages that occur in the literature, and not (necessarily) mutually exclusive settings. The first row is different in kind from the rest: it is not a decision setup an agent might adopt, but a constraint: a set of demands about choosing consistently across decision trees, which then rules certain scoring rules out.
Notes on particular cells. Read them only if you want to know exactly what the names in the table mean; they are not essential for general understanding.
(a) Hammond's requirement: fix a nonempty finite set of states; consider every structurally admissible finite decision tree built over that setup, with strictly positive probability on every chance branch; and require that choices at each information event be generated by consequence-based rules that do not change when the same continuation problem is embedded in a larger tree. Adding continuity then yields an expected-utility representation at each event. The stronger conclusion (one state-independent utility, with subjective probabilities) needs a common consequence domain and further richness assumptions. §5.1 gives the full statement. Removing any of these assumptions can reopen a ✕ cell.
(b) The resolute-and-EU agreement holds under ordinary Bayesian conditioning, a stable utility function, chance that does not depend on which plan was chosen, feasible plans that can be recombined branch by branch, and a consistent rule for breaking ties. §5.2 explains why each is needed.
(c) The §4.2 calculation uses log utility of wealth. That is also the growth-rate rule only if the percentage gains and losses are understood as recurring in a multiplicative process; fixed dollar amounts repeated without scaling would define an additive process instead, where the growth-rate rule ranks by expected wealth.
So, behold! I think it is already quite a useful map, to be honest, even without the claims about the map itself. I think it is at least a useful thing to see what can be studied in decision theory. But I suggest going meta and studying the map itself — more on this is at the end of the post, after I build more examples and intuitions.
3.2 What each proposal actually specifies
The first table asked what happens when a scoring rule is combined with a decision setup. Its rows, though, are bundles: "causal supposition, re-solve at each decision point" fixes two things at once, because that is how the literature packages them. In the second table below, let's do the opposite. We take proposals one at a time and pull each apart into the pieces named in §2 (the rule that scores a prospect, the rule that turns a candidate into a prospect, and the rule relating earlier choices to later ones) so that one can see what each proposal actually settles and what it leaves open.
That last part is the point of the exercise. Many entries in the third column read "not settled by…": they mark places where a theory that gets discussed as though it were complete in fact leaves the question of later choices to be supplied from outside.
So, the prospect-evaluating rule has been studied quite a lot: before 1990, economics and philosophy already contained work by Allais, Ellsberg, Strotz, Machina, Quiggin, Schmeidler, and others, as well as prospect and disappointment theories. LessWrong has focused heavily on supposition rules while usually keeping EU fixed, but Garrabrant's work and infra-Bayesian work are important exceptions.
A rather side note: Buchak's position in the table is a useful example of why the labels about coherence and "valid cells" in the table must remain conditional. Her formal examples mostly use ordinary acts and states. The right dynamic interpretation of REU remains disputed. Thoma, Weatherson, and Pettigrew, Campbell-Moore & Konek (2025) identify pressures from choices over time and from choices about which risk attitude or decision rule to use.
4. Two illustrative calculations
This section is very easy and may be obvious for some, but I thought it may be helpful for others to demonstrate what I mean on a very concrete level. These are just numerical examples that separate the three proposed questions of decision theory by holding some parts fixed and changing others. Of course, each example is local: a numerical verdict in one problem doesn't allow per se to define a complete decision theory. Anyway, feel free to skip if obvious.
4.1 Newcomb's problem: the supposition rule does most of the work
Consider the standard Newcomb setup, in a probabilistic version. Omega predicts whether you will take only the opaque box or both boxes. The opaque box contains $1 million if Omega predicted one-boxing and nothing if Omega predicted two-boxing. The transparent box always contains $1,000. Omega's predictions are 99% accurate, which here means 99% conditional accuracy in either direction: given either contemplated output, Omega matches it with probability
CDT and FDT can agree on all of those stated facts. In particular, they can agree that the opaque box was filled before the choice at the boxes and that physically taking the transparent box adds $1,000. Yet, they still construct different decision-relevant prospects. Standard act-level CDT causally sets the present box-taking act and does not let the earlier prediction or box contents vary with that intervention. FDT varies the decision function's output and, under the stipulated logical dependence, lets Omega's prediction vary with it. The disagreement is about the hypothetical comparison used in deliberation, not about whether the present physical movement reaches backward in time and changes an already filled box.
Suppose the agent has linear utility in money and uses Buchak's risk function
Standard decision-time CDT + REU chooses both boxes for every ordinary REU risk function. In the usual causal model, Omega's prediction and the contents of the opaque box are already fixed before the current choice. If we hold them fixed, taking both boxes gives $1,000 more in every state than taking only the opaque box. Standard REU respects this kind of state-by-state improvement, so the agent's risk attitude does not change the verdict. [23]
A note: let's not argue about whether CDT can be formulated in terms of policies. Merely writing the candidate as a policy does not, by itself, change the intervention point or the result. If CDT instead evaluates an earlier choice to install a one-boxing disposition before Omega predicts, that earlier choice can causally affect the prediction and CDT can endorse installing the disposition. Some authors use policy-CDT for a third construction: ex-ante optimization over whole policies even when no actual commitment was available. That construction can one-box, because the hypothetical policy choice is evaluated from a point causally upstream of the prediction; evaluating from earlier credences alone, with the intervention still at the boxes, would leave the dominance verdict unchanged. The later act-level problem is different, and an agent that optimizes again at the boxes may still two-box. [24]
A local FDT-style supposition + REU calculation chooses only the opaque box. Under the stipulated logical or subjunctive connection:
One-boxing gives $1,000,000 with probability 0.99 and $0 with probability 0.01:
Two-boxing gives $1,001,000 with probability 0.01 and $1,000 with probability 0.99:
So, the evaluation rule still matters: under the same stipulated prospects, a sufficiently pessimistic risk function would return to two-boxing whenever
The supposition rule does most of the work in this version of Newcomb's problem, but it does not do all of it.
4.2 Counterfactual mugging: evaluation matters even after the policy prospect is fixed
Now, Omega, assumed here to predict the relevant policy perfectly, flips a fair coin. On tails, Omega asks you for $100. On heads, Omega pays you $$Z$, but only if it predicted that you would have paid on tails. You wake up and see tails.
An ordinary act-by-act agent that conditions away the heads branch refuses to pay. An updateless formulation instead compares whole policies from before the coin result is observed.
From that earlier point of view, the “pay on tails” policy gives:
The “refuse on tails” policy gives $$0$.
For the first two calculations below, utility is linear in money; the REU calculation changes only the probability weighting. The third calculation instead uses log utility of final wealth.
UDT-style policy choice + linear EU: paying is strictly preferred exactly when
so the threshold is $Z>$100$; at $Z=$100$, paying and refusing tie.
UDT-style policy choice + REU with
Paying is strictly preferred exactly when $Z>$300$; at $Z=$300$, paying and refusing tie.
UDT-style policy choice + log-wealth EU, starting with wealth $1,000: paying is strictly preferred exactly when
The exact threshold is $Z>$1000/9\approx $111.111\ldots$; at exactly $Z=$1000/9$, paying and refusing tie. If payments must be whole cents, the smallest strictly preferred amount is $$111.12$. If these percentage changes in wealth are understood as factors that recur in a stipulated multiplicative process, this is also the canonical multiplicative-EE calculation. Fixed dollar gains and losses repeated without scaling would instead define an additive process, so the EE interpretation requires this extra assumption about the environment.
Now set $Z=$250$. Linear EU and log-wealth EU pay. Under the multiplicative interpretation just stated, EE pays as well. The stated REU rule refuses. So, a policy-oriented supposition rule by itself does not settle the decision without an evaluation rule.
But this example distinguishes linear utility, logarithmic utility, and REU. It does not distinguish fixed-environment EU from EE. To make the latter distinction, we must compare different dynamics; §8.1 returns to this.
5. Compatibility results as an illustration of the entanglement of dimensions
This is probably the core of the article! Initially, I tried to "draw permanent walls between cells": what cells are prohibited, which are allowed, which are open in the map of decision theories. This... didn't work out well. What happened eventually was rather a collection of results with the form:
Four groups of results come from the existing literature. The appendix also gives two AFAIK novel observations, labelled Theorems 7 and 8 there.
5.1 Hammond: demanding consistency across decision trees leads to expected utility
Suppose you insist that an agent's choices "hang together" across every decision tree it could face. Not merely that it ignores branches it can no longer reach — something considerably stronger: that whenever the same continuation problem comes up, whether standing alone or buried deep inside a larger tree, the agent handles it the same way. Hammond (1988) proved that an agent meeting that demand, across a rich enough collection of trees, must rank its options as though it were maximizing expected utility at each point where it has information.
That is a strong argument for expected utility, if you accept the demand. But there are two things it does not establish:
First, it is not a proof that any agent that updates its beliefs and chooses again must use expected utility. The requirement Hammond imposes is much stronger than "learn, then re-decide."
Second, what the argument delivers is local: an expected-utility representation at each information event, taken one at a time. The "conventional" EU theory (a single utility function, together with subjective probabilities, governing every event at once) needs further assumptions on top.
A theory that rejects expected utility can escape the conclusion by rejecting one of the premises, and several theories discussed here do exactly that. But it is not a free escape. A theory that denies a premise owes an account of why the premise was too strong and of what takes its place.
5.2 Rank-dependent rules can reverse later choices; resoluteness blocks one money pump
Why the reversal happens. A rank-dependent rule weights an outcome partly by where it ranks among the gamble's possible outcomes. The worst outcomes might count for extra, for instance. That ranking is a feature of the gamble as a whole. So when information arrives and some possibilities drop out, the survivors can move up or down the ranking and be given different weights. An agent that simply re-applies the same formula to whatever is left can therefore end up preferring, at the later point, the opposite of what it planned. [26]
Whether that counts as irrational doesn't seem to be a settled question, because "dynamic consistency" names several different demands. Must the later action match what the earlier plan specified? Must the later ranking of the remaining options match the earlier ranking of them? Or must the agent merely never end up worse off in every state than it could have been, in which case a reversal matters only if trades exist that exploit it? Theorists with global, nonseparable preferences reject the framing outright: what was assessed earlier is a whole gamble, part of whose value lay in branches that will now never occur, while what is assessed later is a fragment of it, so the two rankings were never required to agree. The reversal therefore settles less on its own than it appears to, at least in academic debates.
What is clear is that it opens a specific vulnerability. Gustafsson constructs a pair of trades that such an agent accepts one at a time, and that leave it worse off in every possible outcome than if it had refused both. Wakker exhibits a related cost: an agent whose preferences violate independence, and who has not committed to a plan, can prefer not to be given free information.
But expected utility avoids it. For an expected-utility agent, scoring a plan now and scoring it after the information arrives fit together automatically. The reason is that averages of averages are averages: the value a contingent plan has today equals the average of the values it will have once the agent knows more. So the plan that looks best in advance has continuations that look best later, which is exactly why the standard method of solving a decision tree backward works.
But there are different protections, available without expected utility! For example, Theorem 7 in the appendix offers a guarantee that does not require the agent to score gambles by expected utility. Informally: look at the whole tree in advance; score complete plans by any rule that always prefers a plan doing at least as well in every state and strictly better in at least one; pick a best plan and carry it out. Such an agent cannot be walked into a sure loss. In particular, if "refuse every offer" was among the plans it considered, no sequence of trades already present in the tree can leave it worse off in every state than refusing would have.
But now, look at what changes between those three findings! In §5.1 the scoring rule gets pushed toward expected utility, but the thing doing the pushing is a demand about decision trees, not a claim about how to score gambles. In the first half of §5.2 a rank-dependent rule walks into a money pump, but only once it is re-applied afresh at each new situation, which is again a fact about the dynamic setup rather than about rank-dependence itself. And then, Theorem 7 closes that vulnerability without touching the scoring rule at all: the same rank-dependent agent, told to fix a plan in advance and carry it out, cannot be led into a sure loss.
The moral is that in each case what decided whether the agent was in trouble was the pairing: how it scores gambles, together with how it relates earlier choices to later ones. That is the claim I state abstractly in the beginning, and here we have concrete theorem examples for it, exhibiting cases in which holding the scoring rule fixed and moving the dynamic setup flips the answer.
Note two consequences:
First, "is this decision theory coherent?" has no answer until a package is specified. When someone says a non-expected-utility rule is incoherent, the correct question is: incoherent paired with what? Paired with naive re-optimization, yes: that is Gustafsson's pump. Paired with ex-ante commitment in a foreseen finite tree, no: that is Theorem 7. Neither answer is a fact about rank-dependence alone.
Second, a theory that rejects expected utility incurs a debt it cannot pay on the evaluation axis. It has to say what happens when information arrives, and then accept whatever that answer costs: Hammond's premises rejected and something put in their place, or commitment together with the standing objection that it acts against what the agent, knowing more, would now prefer. In §7, we will collect those bills.
5.3 An interesting cross-term: when is it good to know things?
Consider an old question: should a rational agent ever pay to avoid seeing free evidence?
Good (1967) proves that free information has non-negative value for an expected-utility maximizer that conditions on what it observes, provided that:
Guess what I am going to say? Each part matters! The value of information therefore depends on like everything: the evaluation rule, the supposition and learning rules, the observation model, and the agent's ability to choose or commit to policies!
Remove any one of them, holding everything else fixed, and the guarantee can fail:
Now notice that the dependence runs the other way as well! Theorem 8 in the appendix changes only the dynamic coordinate (the agent fixes a policy in advance rather than deciding after the signal arrives), and non-negative value comes back for evaluation rules that Wakker's argument would otherwise catch.
That pairing is the heart of it. The same evaluation rule is safe or unsafe depending on the dynamic setup. The same dynamic setup is safe or unsafe depending on the supposition rule. So "can free information hurt?" is not a question about any one of the three, and there is no way to answer it by reading off one coordinate at a time and combining the answers afterwards.
This is a nice illustration of the fact that the three questions are not separable in practice. If they were, one could settle the value of information by looking at the evaluation rule alone, and every result in this subsection would be about expected utility versus its rivals.
5.4 Intersections in Damascus
In Death in Damascus, [32] choosing either city can be unstable: settling on Damascus is evidence that Death predicted Damascus, while switching to Aleppo is evidence that Death predicted Aleppo. Under EU, and assuming that Death cannot predict the result of a private randomizer, a mixed strategy can sometimes be self-ratifying. [33] The ordinary proof uses linearity in mixtures: if the pure actions in the mixture have the same expected utility, then every mixture of them has that value.
So, while two previous subsections before this one paired the evaluation rule with the dynamic setup, this one exhibits a different pairing, and a more surprising one: a solution concept belonging to Question 2 (suppositions) turns out to rest on a property of the answer to Question 1 (evaluating prospects).
The standard treatment of Death in Damascus rescues the agent with a mixed strategy, and the rescue works because expected utility is linear in mixtures. This is not a fact about predictors, about ratification, or about causal reasoning, but a fact about how expected utility scores mixtures. A construction that looks purely dimension-2 is actually leaning on a dimension-1 property.
6. Description and identification
6.1 The same behavior can have several explanations, but different points in decision-theory-space are different
One way to summarize what we saw in §5:
But the second statement seems to be true only within certain limits! Observed choices do not uniquely reveal an agent's internal representation. On a small menu, we can often reproduce the same behavior by changing utility, beliefs, the supposition rule, or the dynamic solution, but there are limits to that. If we go "far enough" in decision-space, the solutions given by the different cells in the map of decision theories seem to eventually diverge. [34]
Also, some cells can explain almost any behavior if interpreted conveniently. For example, EU representation can almost always be restored by expanding the outcomes. For example, the consequence might include disappointment or the menu from which the outcome was chosen. But this changes the model. [35] If we are allowed to put arbitrary action or menu labels into the outcomes, many finite choice patterns can be made to look like EU, and the claim becomes weak and eventually unfalsifiable. A serious proposal to enrich outcomes should still have some restrictions on what counts as different options and use the same rule across all relevant problems.
6.2 Use panels of environments to identify the parts
Different changes to a decision problem test different parts of a theory.
A single decision can sometimes distinguish theories if its menu is rich enough. Allais choices test independence; Newcomb-like choices can test supposition rules when the evaluator and background model are held fixed. What one observed choice usually cannot do is identify a single cell in the map. For that, we need a panel of problems and explicit limits on which alternative representations are allowed.
7. Dropping axioms one at a time: the islands of stability
There is a second, more radical way to generate the space of decision theories. Instead of collecting the theories people have actually proposed, we can start from expected utility's own axioms and ask, one at a time: what happens if I drop this one? The von Neumann–Morgenstern axioms together with the reduction-of-compound-lotteries axiom and its time-domain cousin, stationarity of discounting, [36] each generate a family of theories when relaxed.
Of course, there will be bundles violating any subset of the classical axioms you care to name. The classical literature's implicit position is that nearly all of it is uninhabitable: violate an axiom and you get money pumps, plans dominated by alternatives you could have taken, choices that reverse themselves as information arrives, equilibria that fail to exist. On that view the map needs one continent: expected utility with Bayesian updating. And a warning label for everything else.
But the warning label is too coarse, and I already showed four places where it misleads. This is for structural reasons, I think: every classical coherence argument is conditional. Each axiom is punished-when-violated only by a theorem whose premises include the other classical commitments. Hammond's derivation of independence consumes strong tree-consequentialism and re-optimization at each node (§5.1). Good's theorem on the value of information consumes expected utility, conditionalization, and the assumption that acts carry no information about states (§5.3). Gustafsson's money pump consumes re-evaluation at the later node (§5.2). So a single deviation from the classical bundle sits squarely in the punishment zone, which is what makes the warning label look justified. But correlated deviations can disable each other's punishers, because the punisher for one axiom is a theorem that needs the others.
Like in nuclear physics, there is a valley of stability: the expected-utility column, welcome in every row of §3.1's table. There are decay channels: money pumps, aversion to free information, equilibria that fail to exist, rules that undermine their own use. And there are islands of stability — bundles of mutually protective deviations, each certified against a named channel:
Two caveats, however:
First, there are islands and there are mirages. Some apparent violations are not new territory at all. Final wealth under compounding and the single-level geometric mean both look non-linear and both rank prospects exactly as an expected-utility rule does. Those are not islands but re-descriptions of the mainland, like the same rankings in different notation. An actual island is a bundle that no fixed re-description maps back to expected utility across all the environments in question.
Second, stability is always relative to a channel and always assumptions-based, never absolute. Theorem 7 certifies the resolute rank-dependent island against money pumps in finite, fully foreseen trees; remove foresight and the channel reopens. And a different channel stays open for the same island: Pettigrew, Campbell-Moore & Konek (2025) show that risk-weighted rules can be self-undermining when the agent must choose which decision rule to use, and nothing here closes that.
The same pattern appears on an axis I have not yet touched: time. Stationarity together with dynamic consistency lead to exponential discounting (Koopmans 1960; Strotz 1956). Exponential discounting is to the time axis roughly what expectation is to the risk axis: the form under which re-planning changes nothing.
Each axis has one identity of the same form: evaluating in two steps gives the same answer as evaluating in one. On the time axis that identity is
The failure modes match as well! Every other discount function reshuffles the relative weights of future dates as they approach: the hyperbolic discounter is patient about distant trade-offs and impatient about imminent ones, which is Strotz's preference reversal. That is precisely what every non-expectation rule does to the relative weights of outcomes as information arrives.
And an agent whose rule fails the identity faces the same pair of options on either axis: accept that its later self will overturn its earlier plan, or bind itself in advance so that the later self cannot.
Ulysses took the second option. He knew that on hearing the sirens he would want to steer toward them, so he had himself lashed to the mast, not in order to change what his future self would want, but to ensure it could not act on the want. A hyperbolic discounter equipped with that kind of device is perfectly coherent, and behavioral economics rediscovered the arrangement under the name "commitment devices." In the vocabulary of §3.1, Ulysses belongs in the resolute row: his deviation is on the time axis rather than the risk axis, but the repair is the same one, and it works for the same reason.
My takeaway from this detour is that the "island principle" is not an artifact of risk, or of anything peculiar to lotteries. It is simply what conditional coherence theorems do. Each one names a set of premises, and each can be escaped by giving up a premise and paying for it elsewhere. On whatever axis such a theorem is stated, the same situation appears.
8. Toward a metatheory of decision theories
The three-question framework suggests a research program — call it a metatheory of decision theories: its objects are whole packages (evaluation rule, supposition rule, dynamic solution, learning rule), and its results are the conditional compatibility theorems that link them.
In particular, for any proposed decision theory, record:
At the very least, this bookkeeping blocks several common category errors:
But there may be more to it!
8.1 A gauge analogy: descriptions and invariants
There is a useful analogy here with gauge redundancy in physics and, even more directly, with Klein's Erlangen program in geometry. [39] The basic idea is that several mathematical descriptions can represent the same observable object, so we should ask which features change with the description and which features remain invariant.
Part of that idea can already be stated exactly. Fix:
Let
Thus
I find that this makes the use of the geometric language surprisingly efficient!
There are already several full-fledged examples of description-invariance, old and new:
The EE example also shows why the environment family must be named. In each fixed additive or multiplicative environment, the EE rule has an EU representation. Across both environments on one common wealth domain, [40] no single fixed utility represents the whole pattern. [41] “EU-representable” is therefore true locally and false uniformly. That exact failure can motivate the picture of local descriptions that do not combine into one global description.
Still, the "gauge representation" may be a real thing but may also be just an analogy. But the analogy is still useful because it changes the questions we ask:
This is close in spirit to the Erlangen program: study a space through its transformations and invariants.
My previous post argued that the independence axiom is not the boundary of rationality. This post has tried to say what the territory beyond the boundary is actually shaped like: not a free product of independent choices, but a constrained space whose constraints are theorems. The zoo has a geography: a mainland, a sea whose currents are the coherence theorems, and islands of correlated deviations that certify each other's safety. I have tried to draw the first map coarse enough to be checked and corrected.
And if the zoo has a geography, the natural next question is what the terrain is made of — what the space of decision theories actually is, mathematically. There are hints scattered through this post that the answer is algebraic. Klein's Erlangen program characterized each geometry by its transformation group and its invariants. Whether the metatheory of decision theories admits an Erlangen program of its own, whether the zoo has not only a geography but an algebra, is an interesting question to ask.
The independence axiom says that if an agent weakly prefers lottery
A prospect is the uncertainty-bearing description of payoff-relevant consequences that a theory uses to evaluate a candidate act or policy. In the simplest and most common case, it is one probability distribution: “$100 with probability
Utility is a numerical representation of preferences; it need not mean pleasure or money. Expected utility (EU) evaluates a lottery
A weak ordering is a ranking that is complete and transitive: every pair can be compared, and if
Rank-dependent utility (RDU) changes the decision weights according to an outcome's rank in the gamble, rather than multiplying each outcome's utility by its probability directly. Risk-weighted expected utility (REU) is Buchak's formulation within the same broad rank-dependent family, using a risk function
A supposition rule defines a controlled hypothetical comparison. It identifies what is being supposed—for example, learning an act, causally setting it, or varying a computation's output—and determines what remains fixed and what changes with that supposition. A counterfactual asks what would be true if something were different. A subjunctive dependence is a broader “would-change-if” connection. It may be causal, but in FDT it can also be logical: varying the output of a computation can vary the modeled outputs of predictors or identical copies, even when the computation does not physically cause their earlier behavior. ↩︎
Conditioning replaces a probability distribution with the distribution conditional on an event, such as “I choose
A common cause influences both an action and an outcome without the action causing that outcome. In a simple smoking-lesion model, a lesion causes both smoking and cancer, while smoking itself is stipulated not to cause cancer. Whether EDT treats the contemplated act as additional evidence depends on what the agent already knows. In “tickle defence” versions, present evidence about the internal urge or decision process can screen off the act from the lesion, so EDT need not avoid smoking. The main text uses a version without that screening-off assumption solely to illustrate the difference between conditioning and intervention. ↩︎
Ex ante means evaluated from an earlier point of view, before a specified observation or event; ex post means evaluated afterward. An agent may update its beliefs after observing without abandoning a previously chosen plan, so “updates beliefs” and “optimizes again” must not be treated as synonyms. ↩︎
A representation result shows that a pattern of preferences or choices can be described by a particular mathematical form, such as expected utility. Dynamic consistency concerns agreement between plans made earlier and choices preferred later, once the relevant conditioning is specified. An equilibrium is a stable set of choices in which no participant—or no relevant future self—has an incentive to change unilaterally. Coherence is broader and potentially ambiguous; throughout the article, it should be read only through a named property or theorem. ↩︎
The Single Choice Principle (SCP) says, roughly, that making a conditional plan for one choice in advance and making that same choice when the condition is reached should give the same verdict. Its motivation is a Ramsey-style test: to ask what you should do if
Ratificationism adds a stability test: an option is rationally available only if, after supposing that the agent will choose it, no other option becomes better by the theory's own standard. In predictor cases, a contemplated choice can change the evidence about what was predicted, so simple maximization can oscillate. A ratifiable choice does not undermine itself in this way. Causal ratificationism evaluates the candidate options by their causal consequences while applying this self-endorsement test. Depending on the model, more than one choice or a randomized choice can be ratifiable. ↩︎
A preference is additively separable across components if some numerical representation of its ranking has the form
A policy is a complete contingent plan. A pure policy is a function from possible observation histories to actions; it may say “choose
A representation theorem states conditions under which some abstract preferences or choices can be represented in a specified mathematical form. It does not automatically provide the correct causal model, solve zero-probability conditioning, or explain how the representation should be used at later decision points. ↩︎
A Cartesian product is the set of all combinations formed by choosing one item from each of several sets. Here it would mean every possible triple of one evaluation rule, one supposition rule, and one dynamic solution. The article does not claim that every formally writable triple is a complete or coherent theory. ↩︎
Ergodicity economics (EE) connects decisions to the long-run growth rate of a quantity such as wealth under repeated dynamics; Peters and Adamou's time interpretation of expected utility explicitly presents the correspondence between dynamics-specific growth optimization and expected utility. In the canonical cases used here, stationary ergodic additive increments with suitable finite averages lead to maximizing expected wealth change, while positive stationary ergodic multiplicative factors with finite expected log lead to maximizing expected log-wealth change. For a specified process and measured quantity, ergodicity means, roughly, that the long-run average along almost every path agrees with the corresponding average across the probability distribution, when the relevant averages and limits exist. The article uses only these idealized additive and multiplicative prescriptions; it does not assume that every claim made under the EE label follows from them. ↩︎
An ordinary state-contingent act is represented as a mapping from possible states to consequences. The ambiguity papers placed in this row study preferences over such acts and learning rules for updating them. That framework does not, by itself, say whether Newcomb-like or common-cause cases should use causal intervention, evidential conditioning on the act, or a logical/subjunctive comparison. This is why the row is an updating package rather than a clean value of
A multiple-prior model represents ambiguity with a set of probability distributions rather than one distribution. Rectangularity is a structural condition that lets the set of priors be broken into compatible marginal and conditional pieces across information stages. A recursive rule defines today's value partly through the values assigned to possible continuation problems. These details are what make the relevant consistency result work; “uses several priors” by itself is not enough. ↩︎ ↩︎
The Kelly criterion chooses bets to maximize expected logarithmic growth of capital in a repeated multiplicative setting. A maxmin expected-utility rule represents ambiguity with several candidate probability distributions, calculates expected utility under each, and judges an act by the lowest of those values. Unrestricted maxmin updating can disagree with its earlier plan; special structures or updating rules are used to recover particular forms of dynamic consistency. ↩︎
Infra-Bayesianism is a family of attempts to generalize ordinary Bayesian models so that an agent can reason about richer uncertainty and about environments that may respond to its policy. It often uses sets of probability-like models or generalized expectation objects rather than one fixed probability distribution. The exact formalism varies. Calling a proposal policy-based does not by itself show that it uses FDT-style logical supposition or an updateless dynamic solution, so the table records only a related family rather than placing every infra-Bayesian proposal in the row. ↩︎
A deliberational decision theory does not simply maximize once under a fixed supposition about its own action. It looks for a recommendation that remains stable after accounting for what the agent's contemplated choice says about the decision problem. Ratification theories are important examples, although the exact use of the label varies across authors. This is a within-deliberation condition, not by itself an answer about how choices at different histories should relate. ↩︎
Statewise dominance means that one act gives at least as good an outcome in every possible state and a strictly better outcome in at least one state that has positive probability. A rule that respects it never chooses the dominated act solely because of its attitude to risk. In standard act-level Newcomb, two-boxing gives exactly $1,000 more for each fixed content of the opaque box. ↩︎
Writing candidates as policies is not the same as ex-ante policy optimization. Three constructions should be separated. First, Everitt, Leike, and Hutter define a sequential policy-causal intervention and prove, in their model, that it is equivalent to intervening on the current action; this version still gives standard CDT's two-boxing verdict. Second, an actual earlier commitment or disposition choice made before Omega predicts can causally affect the prediction, so CDT may favor installing a one-boxing disposition even though a later uncommitted CDT agent two-boxes. Third, Oesterheld and Conitzer use policy-CDT for following the whole policy that CDT would have preferred to commit to ex ante, even when no actual commitment was available; their ex-ante-commitment interpretation one-boxes in their Newcomb variant, where an earlier decision point precedes the predictor's observation — a dependence they note explicitly. Thus the familiar objection that policy-based CDT one-boxes is correct if “policy-based CDT” means this third proposal, but not if it means that the candidates in standard decision-time CDT have merely been written as policies. The article uses the mere-policy-representation setup unless it explicitly says “ex-ante policy optimization.” ↩︎
Tree-consequentialism, in the strong sense used here, requires choices in a decision tree to be generated by consequence-based choice rules that remain invariant when the same continuation problem is represented in different larger trees. Hammond first fixes a nonempty finite state set
A conditional preference reversal occurs when an earlier plan ranks one continuation above another, but after learning which branch was reached the agent ranks the continuations in the opposite order. Such a reversal is not automatically irrational: whether it counts as dynamic inconsistency depends on which earlier and later objects the theory says should agree. ↩︎
A money pump is a sequence of individually accepted trades that predictably leaves the agent worse off, often with a sure loss. Money-pump arguments are conditional: they require a specified sequence of offers, an account of what the agent knows, and assumptions about whether it can commit or refuse. ↩︎
Gustafsson's §5.2 starts from the weak strict-preference failure of independence
A preference is complete if every pair of options can be compared and transitive if
The tower property says that taking a conditional expectation and then averaging over the possible information states gives the original unconditional expectation:
A partitional observation tells the agent which member of a mutually exclusive and exhaustive partition contains the true state. An inexact or non-partitional observation need not have this clean form. Good's result also assumes that observing does not change the payoff-relevant world, apart from changing what the agent knows and which contingent action it selects, and that the original act menu remains available after every signal. ↩︎
In Death in Damascus, Death predicts where the agent will go and waits there. If the agent settles on Damascus, that is evidence Death predicted Damascus; if the agent switches to Aleppo, the same reasoning points to Aleppo. The case is used to study unstable deliberation, ratification, and whether private randomization can help when the predictor cannot foresee the randomizer's result. ↩︎
A mixed strategy deliberately randomizes among actions. A choice is self-ratifying if supposing that it will be chosen does not make another choice better. A Nash equilibrium is a profile of strategies in which no player benefits by changing strategy alone, given the others' strategies. Standard existence results normally assume an expected-utility treatment of mixtures or related conditions that make preferences over mixtures sufficiently regular. ↩︎
But I am not sure. Maybe not, and there are actual full "invariants". See however section 8. ↩︎
Outcome enrichment means redefining consequences to include features that were previously left out, such as regret, disappointment, the menu, or the action label. This can make a formerly non-EU pattern representable as EU, but only by changing what the theory treats as an outcome. ↩︎
A time-separable discounted-utility model adds independently valued rewards from different dates after multiplying each by a discount factor. Stationarity says that preferences between time patterns do not depend on a common shift in calendar time. Under familiar extra assumptions, and after choosing the normalization
A menu-independent evaluator assigns a value to each available policy's prospect without changing that prospect's value when other policies are added or removed. Prospect extensionality means that two policies producing the same decision-relevant prospect receive the same value. A constant policy ignores the new signal and chooses the same old act after every signal. If every old act has an outcome-equivalent constant policy, the new menu contains a copy of the old menu. ↩︎
Incomplete preferences allow some pairs of options to remain incomparable. They cannot in general be represented by one real-valued score, because a single real number for each option automatically orders every pair, allowing ties. ↩︎
Klein's Erlangen program classified geometries by the transformations under which their central properties remain invariant: Euclidean geometry studies what rigid motions preserve, for example. In physics, a gauge redundancy occurs when different mathematical descriptions represent the same observable physical state. An orbit is the set reached by applying a specified family of composable transformations to one object. A chart is a coordinate description of part of a space with compatible overlap rules, and curvature requires additional geometric structure for comparing directions at different points. This article defines a behavioral equivalence relation and identifies some exact behavior-preserving redescriptions. It does not yet supply the general transformation system or geometric structure required to use all of these terms literally. ↩︎
A common outcome domain means that the same set of consequences is used across the compared environments. If the environment label is added to each consequence, “£1,000 in environment
The von Neumann–Morgenstern uniqueness result says that two expected-utility functions represent the same non-trivial preference over a sufficiently rich lottery domain only if one is a positive affine transformation of the other: