This is the write-up of my MATS project so far, as a mentee of Abram Demski. Those who would prefer a more formal/academic/mathematical write-up should look at the AAMAS submission here.

TL:DR, a formal framework for representing CDT, EDT and UDT on the majority of classical Newcomblike problems, plus some accompanying results about how CDT can be made to pay for ignorance.

Introduction

I present my framework for hosting classical decision theories (Causal, Evidential and Updtateless/Functional decision theories) in classical Newcomblike problems. The framework is intended to be faithful (the outputs of the framework regarding what theories act in which ways in which problems agree with the consenus), fair (each theory's intuitions about "why this is rational behaviour" can be expressed as formal properties of the framework, the framework tries to avoid "begging the question" in favour or against any particular theory) and computable (given a formal description of any theory and any problem (at least those that the framework can express) there is a simple algorithm do determine what that theory would do in that problem. To the best of my knowledge, there is no other framework that fulfills these three properties while hosting CDT, EDT and UDT.

We can broadly divide Newcomblike problems into two categories: read problems (the world varies with the agent by reading the agent, e.g Newcomb's Problem) and write problems (the world varies with the agent by writing some portion of the agent, e.g Smoking Lesion). My framework only handles read problems, rejecting write problems mostly as a consequence of giving agents perfect introspection.

The most significant finding from my framework thus far is that a calibrated^[1] CDT agent will in some cases pay to receive reduced information, and objectively benefit for doing so. To the best of my knowledge this is an orginal result.

The Framework

Decision Spaces

A Decision Space is comprised of the following elements:
• 𝑆 is a finite set of states containing the “invalid” state 𝑠0. The state set is partitioned into start states and end states, . Since this framework is concerned only with single-step problems, this split encompasses all possible states.
• 𝐴 is a finite set of actions.
• Acc : 𝑆′ → 2𝐴 \ ∅ is the accessibility function, assigning to
each start state its available actions.
• 𝛾 : 𝑆′ × 𝐴 → 𝑆′′ is the outcome function, such that .
•  is an observability set that partitions 𝑆′, such that. It follows that we may write Acc(𝑜) without ambiguity.
• 𝑈 : 𝑆′′ → R is a utility function.

Predictor Mechanism

𝐶 is a predictor mechanism, written as a tuple 𝐶 = (1−𝜀, 𝑜′, 𝐹 ). 1 − 𝜀 is the predictor’s reliability, 𝑜′ the predicted observation (the observation that the predictor "cares about"), and 𝐹 a mapping from the realised prediction to a distribution over start states, such that:

A Decision Space is a tuple 

Decision Policy

Given a decision space  with observation set  and action set , a decision policy for  is a function  such that:

Let  be the set of all decision policies for .

Predictors

An important property of predictors in Newcomblike problems is that they are decision-sensitive, meaning that the prediction is correlated with the agent's eventual realised action over any other property of the agent. Correspondingly, the predictor reads (with some random noise) the policy's action at the predicted observation o'.

Let  be the set of all  pairs where  is a decision policy for , for all possible .

The predictor function is a map  such that for  with  from  and ,
 is the distribution on :

By the construction of the preidctor mechanism and the predictor function, we ensure that any decision procedure encountering any (expressible-within-the-framework) problem has a greater-than-zero chance of being in any of the possible start states. This is mostly acheived by enforcing that the predictor can be arbitrarily close to perfectly accurate, but cannot actually be perfectly accurate.

This choice avoids phenomena such as an agent's action at a particular observation rendering that observation logically inconsistent with the agent (and thus making the choice of action difficult to analyse) or an agent making a certain observation and concluding with logical certainty how they are about to act. I am reasonably condifent that truly perfect prediction is probably impossible in practice, so I do not consider this a high price to pay.

Decision Problems

Given , a nonempty set  is a decision problem for .

There is some debate over whether it is legitimate to consider only some sub-branches of a decision space when comparing two theories on that space. For maximum flexibility I allow this, though I do also formalise why we might want to say that some such comparisons are illegitimate (mostly if the two theories reach those branches at very different rates).

Agent Beliefs

Agents believe that the problem functions exactly as described by . They are not required to hold any beliefs about the explicit probability of themselves performing certain actions. 

Note that the agent only has access to the predictor through its description in , which gives only the parameters of the predictor (accuracy, predicted observation, response to prediction) and describes nothing of how the predictor works or what it will actually predict. The predictor, on the other hand, has direct access to the decision policy of the agent and its output on . This prevents circular dependency or the agent trying to "game" the predictor. It also preserves the decision-sensitivity of the predictor.

Epistemologies

Since agents do not (necessarily) have any beliefs about the action they will take, nor any beliefs about what exact action will be predicted, the agent cannot straightforwardly evaluate the probability of being in any particular start state or the expected utility of the various accessible actions. An epistemology is the device that fixes this gap. Different epistemologies emit different data shapes as required for different decision procedures.

The following list of epistemologies is not considered exhaustive, but is what is required to formalise the theories I want to study.

Prediction-Skeptic Epistemology

Prediction-Skeptic epistemology refuses to accept and/or make use of the decision-sensitivity of the predictor. In particular, Prediction-Skeptic epistemology holds that it is appropriate to determine about your beliefs regarding what the prediction is likely to be, and then to reason about what action you should take without ever updating your beliefs about the prediction.

For simplicity, I model Prediction-Skeptic epistemology as always having uniform uncertainty over what the prediction could be^[2], though a more complete model would allow for more variety in Prediction-Skeptic beliefs.

Its output is a vector over  representing the (presumed) expected utility of each accessible action. Its generative law is the uniform mixture ; conditioning on  gives the
within-cell posterior . For each a \in \Acc(o) emit

Fully Transparent Epistemology (Updateful)

Fully Transparent epistemology accepts decision-sensitive predictions (in particular, the notion that you cannot separate reasoning from what the prediction was from reasoning about what you will do), and believes that an agent can meaningfully and reliably reason about how they would act given observations they did not make (this is especially important for when the agent is not at the predicted observation ).

Its output at  is a matrix over  which represents "the expected value of  if I knew I was about to do ". EDT cares only about the diagonal but CDT needs the whole matrix.

For each supposed resolved action , the credal state is , with start-state law ; conditioning on  gives . For each actual act a', emit

It's output at  is a vector over  which represents "the expected value of , given that I know that I am the kind of agent who would do  at ".

Query the paired theory for its resolved -action . The generative law is
, and for each  emit

.

Partially Transparent Epistemology (Updateful)

Partially Transparent epistemology also respects decision-sensitivity, but is not confident in its ability to reason about how it would act at observations not made. At  it outputs the same as Fully Transparent, off  it outputs the same as Prediction-Skeptic.

Updateless Epistemology

Does not query which observation was made, simply outputs the expected utility (over  as a whole) of each possible decision policy . The output decision policies do read the observation, so it is never in danger of (for example) selecting an action that is not accessible.

Outputs a vector over . For each policy , emit:

where  is the unique observation cell containing .

Decision Procedures/Decision Rules

Each Decision Theory must be formalised in this rule as a Decision Procedure which is the combination of an Epistemology (above) with a simple Decision Rule (below).

Once again, the list below is not claimed to be exhaustive. Due to the differently-shaped outputs, different decision rules can only accept certain epistemologies, as denoted in the square brackets.

Prediction-Skeptic CDT - [Prediction-Skeptic]

.^[3]

Pick the action with the highest expexted value according to the epistemology.

Naive Ratificationist CDT - [Fully or Partially Transparent]

At , call  ratifiable iff its diagonal entry is its own row-max:

Equivalently, an action is ratifiable iff, conditional on knowing you are about to perform , no other accessible act beats . This ties in to the ratifiable causalist argument that while you may use your imminent action as evidence about the world, it is not rational to use your action as a lever on the world via your beliefs.

Then choose best ratifiable action 

If no action is ratifiable, the rule returns ^[4].

Off , , using the
off-cell vector supplied by the paired epistemology (self-knowledge
 under Fully Transparent; uniform random  under Partially Transparent). This also returns  if there is no ratifiable action at .

EDT - [Fully or Partially Transparent]

At , argmax over the diagonal:

Unlike CDT, EDT claims that the appropriate way to calculate the expected utility of an action is "if I knew I was going to perform , what would be my expected utility?" Thus EDT uses the same matrix as NR-CDT, but only cares about the diagonal. This follows the EDT intuition that "the best action is the action that maximises my expectation of utility, conditional on performing that action". 

Off , same as NR-CDT: .

UDT - [Updateless]

, where 

This is closest to a formalisation of UDT 1.1, since the agent picks an entire policy for  (and is assumed to be able to reliably pick the same policy any time it encounters ).

The crucial and quintessential property of updateless decision theories is that the initial choice of policy is made without consulting the observation, selecting whatever policy does best overall. This leads to the property that UDT is unbeatable (in expectation) on any decision space , and also to the property that a UDT agent will sometimes voluntarily sacrifice utility for a benefit that that instantiation of the agent will never see.

Evaluation

We evaluate decision theories on Decision Problems rather than only on Decision Spaces. This is a strictly broader notion as, in this framework, a Decision Problem can be any subset of the observation space of a Decision Space, including the entire Decision Space. The main reason for this choice is the desire to create a framework that, as far as possible, is "fair" to all decision theories in that it allows their favoured arguments and counter-arguments to exist as formal properties of the framework, and tries not to rule those arguments invalid by definition.

Permitting evaluation only at the level of Decision Spaces would provide a clear argument in favour of UDT, as (provably) UDT can never be beaten at this level, and strictly beats every other formalised theory in at least one well-known problem. However, this argument would be won by side-stepping, rather than directly addressing, the updateful argument that UDTs willingness to voluntarily burn to death is a meaningful downside.

The framework does also contain the means to argue that bomb-branch style comparisons are unfair in some way, by the notion of "comparison-calibration".

Calibration

I define three notions of calibration. The first two relate to agents within decision spaces/problems, and the third compares two agents within a decision problem.

Observation-Calibration

A decision procedure is observation-calibrated on some decision problem  if its beliefs about what state it is likely to be in (conditional on the observation) are correct.

Generative-Calibration

A decision procedure is generative-calibrated at  if its beliefs regarding how likely it is to be in any particular state (not conditioning on the observation) are correct. In other words "how likely is an agent like me to find themselves in any particular state if they were to play this game from scratch?"

The Fully Transparent and Updateless epistemologies are observation- and generative-calibrated everywhere all the time (Updateless epistemology is always observation-calibrated even though it arguably doesn't care about being so). Partially transparent is always observation- and generative-calibrated at  (the predicted observation) and mostly not calibrated otherwise^[5]. Prediction-Skeptic epistemology is mostly not calibrated^[5].

Comparison-Calibration

Comparison-calibration is measured on a set of Decision Procedures (typically 2) relative to a Decision Problem  in Decision Space  and it measures the ratio of how frequently each Decision Procedure encounters that specific Problem in  (or the ratio of the most and least frequent if comparing more than 2 Procedures).

Worked Examples

Newcomb's Problem

I tweak some of the numbers here to make the mathematics a little simpler, but this is scructurally Newcomb's Problem.

You find yourself in a room with two boxes, one small transparent box and one large opaque box. By some reliable method you learn the following facts:

• The small box definitely contains $1.
• The large box contains either $1 or $10.
• The big box is completely opaque, soundproof etc. there is no way to work out how much money is in the big box without opening it.
• By some fancy mechanical system, you must either open only the big box (1Box) or both boxes (2Box).
• One week ago, a highly accurate predictor predicted what you would do. If they predicted you would 1Box they put $10 in the big box, and if they predicted you would 2Box they put $0 in the big box.
• The predictor has played this game many times before, though any individual person only plays the game once. The predictor is consistently 99% accurate at predicting the player's final decision. Even against people who try all manner of psychological tricks or last-minute changes-of-mind, the predictor is 99% accurate.
• Nonetheless, the amount of money in the large box was fixed a week ago, shortly after the prediction was made. There is no hidden trapdoor in the big box, and no temporal wormhole for the predictor to peek through.
• The process by which people are invited to play this game is uniformly random, so the kind of agent you are/the decision you would make has no bearing on how likely you are to encounter this problem in the first place.

• • There are two relevant starting configurations for the problem, eiter the big box contains $10 () or $0 ().
• • Four possible end states, plus the invalid state.
• , with 
  • Two possible actions, which are both always accessible.
• • The big box is opaque, so the agent cannot distinguish which start state they are in, only that they are in one of the two possible start states.
•  ,
         
         
         
  • The big box being full is worth $10 (since you always open the big box), the big box being empty is worth $0, and opening the small box is worth $1.
•  with
  ,
  , all else .
  • The predictor fills the box or not with certainty based on its prediction, but the prediction is only 99% accurate.

Verdicts - PS-CDT

Prediction-Skeptic CDT straightforwardly concludes that it will get an extra dollar by two-boxing, and two-boxes. Note that since two-boxing causally dominates one-boxing for this problem, the exact prior beliefs about the prediction do not matter, as long as they are held independent of the agent's action.

Verdicts - NR-CDT

The argument here goes something like "One-boxing is irrational because if I know I am about to one-box, my beliefs about the likely state of the world will tell me that two-boxing would be better, making it irrational to actually follow through and one-box. On the other hand, if I know I am about to two-box, I should still two-box, therefore two-boxing is rationally defensible".

Verdicts - EDT

"If I know I am about to one-box, then I expect to receive far more money than if I know I am about to two-box, therefore one-boxing is much better news than two-boxing, therefore I should one-box."

Verdicts - UDT

"The overall best-performing agents are one-boxers, therefore I should be a one-boxer."

Transparent Newcomb

Transparent Newcomb is extremely similar to Newcomb's problem, the main difference being that the large box is also transparent, clearly displaying how much money it contains. This also leads to the secondary change that the predictor only cares about what you would do if you saw $10 in the big box (this is to prevent "impossible-to-predict" agents who would always do the opposite of the revealed prediction).

The formalisation is identical to Newcomb's problem except that:

•   with 
  • The agent can now clearly identify eactly which start state they are in, and the predictor only cares about that the agent does in the  observation.

Verdicts - PS-CDT

A Prediction-Skeptic agent might appreciate the increased certainty, but otherwise not much changes for them. +$1 is still +$1.

Verdicts - NR-CDT

Here the argument that "if I know I am about to one-box, then I know that I would be better off two-boxing" is even more straightforward given the transparency of the big box.

Verdicts - EDT

This is the one theory that flips on Transparent Newcomb, since the evidence of "I can clearly see $10 in the box" outweighs the evidence of "I know I am about to two-box against a decision-sensitive predictor".

Verdicts - UDT

Here "R1P2" means "one-box at  and two-box at " and so on. UDT uniquely continues to one-box at  (the observation it makes most of the time) since (before conditioning on observation) one-boxing at  vastly out-performs two-boxing there. UDT freely two-boxes at  since the predictor does not care about that observation.

Results

Optimality Conditions

This formalises the idea that "UDT cares about decision spaces, EDT cares about observations and CDT cares about states".

• (Ex ante  UDT) UDT attains  over all policies in .
  • That UDT always obtains the maximum valuation in any given space follows immediately from the definition of UDT in this framework.
• (Interim  EDT) EDT (either transparency variant) attains .
  • EDT always acheives the maximum possible value at the predicted observation (conditional on making that observation), but sometimes loses out to UDT at non-predicted observations as UDT is willing to sacrifice utility at the predicted observation in order to benefit those observations, whereas EDT is not.
• (Ex post  CDT) If some  strictly dominates on an observation , then PS-CDT and NR-CDT select  at , and attain  at every .
  • Since, in non-singleton observations, agents cannot observe which state they are in^[6], the only way to guarantee picking the optimal action for their state is to pick the dominant action (if one exists).

Equivalence Conditions

Various decision theories recommend the same action on some problems. For example, it is generally accepted that EDT and UDT agree on Newcomb's problem (both one-box) and that EDT and CDT agree on Transparent Newcomb (both two-box). I can prove some general rules about when various decision theories agree. I only claim that these results hold in the simplified world of my framework, not necessarily in all conceivable Newcomblike problems.

Singleton Observation  UDT = EDT

If  contains only a single starting observation (meaning ), then UDT and EDT endorse exactly the same set of policies up to tie-breaking. This is because a single starting observation in some sense forces EDT to be updateless, because the agent gains no information about where they are in the problem before making the observation, therefore the optimal policy before making the observation (UDT) is the optimal policy conditional on making the observation (EDT).

Note that this result alone shows that the (tickle-free) Smoking Lesion cannot be faithfully represented in this framework (at least according to the view that EDT refrains from smoking in tickle-free Smoking Lesion while UDT smokes). This is because if the agent perceives the world in exactly the same way regardless of being affected by the lesion or not (i.e one starting observation) then EDT and UDT must recommend  the same action. I argue in "Write Problems" that this is at least partly because Smoking Lesion is not clearly well-formed for perfectly introspective agents.

Singleton Predicted Observation   EDT = CDT

If the predicted observation o' is fully observable (meaning ) then EDT and CDT (under the same transparency variant) endorse exactly the same set of policies up to tie-breaking, and PS-CDT agrees with both at . This is because observing  provides 100% certainty about which state the agent is in, meaning that, conditioning on observing , the chosen action informs expected utility only through  and  (the predictor has no influence after conditioning on  as  is now known) which both EDT and CDT respect equally.

Given that they act the same at , EDT and CDT (with matched transparency) will also act the same at all other observations, since either their beliefs about the prediction agree (fully transparent) or they both assume the prediction is uniform random (partially transparent).

Not Decision-Sensitive  UDT = EDT = CDT

If  is not decision-sensitive (every possible policy sees the same distribution over start states). Then UDT, EDT and CDT all endorse the exact same policies. This is because all three theories are happy to fix their beliefs about the probabilities of states before reasoning about how to act, and all agree exactly on the probability distribution over states. Therefore acting to maximise expected utility treating the state, observation or decision space as fix all produce the same results.

This result should not be read as "decision-sensitive predictors are the only phenomemon where UDT, EDT and CDT disagree" but "decision-sensitive predictors are the only phenomenon formalisable in thsi framework" where the three disagree.

Formalising the Canon

This table collects the results of formalising seven well-known Newcomblike problems in my framework. It lists the responses 

The formalisation of each of these problems can be found at the bottom of this post (besides Newcomb and Transparent Newcomb, which can be found above).

Write Problems

To my understanding, the key and defining feature of Newcomblike problems is that they force some significant feature of the problem to be statistically correlated with the eventual decision of the agent (in at least one branch of the problem) despite said feature being causally prior to the agent's decision. This can be done by having the agent's eventual decision influence the initial state of the problem (typically through decision-sensitive predictors) (read problems) or by having the initial state of the world influence the agent's eventual decision (write problems).

To be properly Newcomblike, the influence in write problems should in some way sidestep the normal operation of the agent's Decision Procedure, otherwise every decision problem where the agent learns something would technically be a Newcomblike problem, since the starting observation in the problem does influence the agent's final action.

Unfortunately I cannot represent write problems in my framework, though I think that the reasons why are themselves instructive. To illustrate them, I will use the example of Smoking Lesion.

Smoking Lesion

The problem of Smoking Lesion can be described roughly as follows:

• In a parrallel universe, smoking has no negative (causal) impacts on a person's health.
• However, there is still a strong correlation between smoking and getting cancer. This is because of a particular genetic condition (the "smoking lesion") that causes people to be predisposed towards both smoking and cancer.
• You are an inhabitant of this universe deciding whether you should smoke.
• You would like to smoke, you don't want cancer, and you would prefer not smoking without cancer to smoking and having cancer.

The classical consensus on this problem is that CDT and UDT smoke freely, and that EDT refuses to smoke because of the correlation between smoking and cancer.

The Tickle Defence

If we assume that the smoking lesion works by producing a strong desire to smoke, or by making smoking more appealing or enjoyable, then we can argue that EDT also smokes freely. This is because upon observing within itself the desire to smoke, the EDT agent then learns everything it could learn about how likely it is to have the lesion, meaning that its eventual decision to smoke or not will carry no further information about how likely the agent is to have the lesion, meaning that the agent can smoke freely.

This version of Smoking Lesion is quite easy to model:

•  
•  ;
• , with ;
•  (the agent can see which sub-game they are playing);
• : ;
•  with  and .

However, defined in this way Smoking Lesion (tickle defended) is not a "true Newcomb problem" because while there is a very strong correlation between choosing to smoke and cancer, this enters entirely through the agent's algorithm in the standard way (I see that smoking gives +$5 utility, I see no reason not to smoke, I smoke).

Tickle-Free Smoking Lesion

If we want the classical Smoking Lesion verdicts (where UDT and CDT smoke and EDT doesn't) we could imagine that the lesion works in some other way, that does not alert the agent (so that EDT treats the decision to smoke as evidence of the lesion). For example, perhaps the lesion induces some completely invisible (to the agent), subconscious bias towards smoking.

However, if the lesion and its effects are entirely invisible to the agent, this would mean that UDT and EDT must agree on the verdict (see the results above), meaning my framework cannot host the tickle-free smoking lesion with its classical verdicts. This is unsurprising as I have no mechanic that could simulate a subconscious bias, but it is also not clear how such a bias would work.

I couldn't model the lesion's effect by biasing the agent towards a certain decision theory, since an agent could then get evidence on whether it has the lesion by considering what kind of decision theory it has. Furthermore, I want the framework to be "theory-neutral" to the extent that I can drop in any kind of agent and meaningfully compare them on the "same" problem^[7].

I couldn't model the lesion's effect by having some probability of forcing the agent's final output, since then the output of the EDT algorithm carries no information about whether the agent has the lesion, only whether or not that output is ignored does.

I think that the statement of Tickle-free Smoking Lesion is at least partially incoherent. We state "there is a statistical correlation between choosing to smoke and having the lesion" and then ask "what would UDT/EDT/CDT do?" This question seems to imply a deterministic answer to each of these questions, which is only compatible with the correlation if we assume that EDT agents never have the lesion. But then knowing oneself to be a follower of EDT would be extremely strong evidence against having the lesion, as outlined above.

That said, it does appear that invisible (or near-invisible) subcoscious biases can exist in humans, which suggests that Tickle-free Smoking Lesion is coherent at least for human-like agents. I think the main reason for this is that humans are not perfectly introspective agents. Any agent with perfect introspection^[8] should be able to identify the presence or absence of whatever means the lesion uses to influence the final decision, meaning that an EDT version of such an agent will never gain additional information from observing its final decision and can freely smoke.

Therefore the only ways to make this work as far as I can see would be to remove perfect introspection from the agents, or to allow the agents to have very strong (possibly Knightian) uncertainty about how the correlation in Smoking Lesion actually works. Either of these would require interesting but non-trivial adaptations to the model. 

Value-of-Information

EDT Will Pay for Ignorance

A well-known criticism of EDT (especially from academic Causal Decision Theorists) is that EDT is inherently irrational because it acts more in the manner of "managing the news" than actually trying to get good things to happen.

One of the most damning (at least according to EDT's critics) examples of this is EDT's willingness to pay considerable amounts of utility to not be told free information in certain Newcomblike problems.

For example, suppose that an EDT agent is about to enter a room where a game of Transparent Newcomb has been set up (the prediction has already been made). Suppose that someone outside the room offers the agent a blindfold, preventing them from seeing how much money is in the big box. If the blindfold is "visible to the predictor"^[9] then wearing it essentially turns the game into Newcomb's problem. If we use the formalisations above, a fully-calibrated EDT expects to gain $9.90 from Newcomb's problem but only $1.10 from Transparent Newcomb (correctly for both problems). Since choosing to wear the blindfold would influence the EDT agent's beliefs about how it would act in the room and hence how much money is ultimately likely to be in the box, it appears that an EDT agent would not only wear the blindfold but would pay up to $ for the privelege.

While they disagree on how you should actually act in Newcomb's problem and Transparent Newcomb, both UDT and CDT agree that the two problems are largely equivalent and that you should mostly perform the same action in both problems^[10]. In fact, UDT would pay a very small amount of utility to refuse the blindfold so that it can freely two-box if it sees the large box empty (which happens 1% of the time). There seems a fairly wide consensus that "pay $ for reduced information" is an accurate description of what EDT would do, but it is shared mostly amongst Causal and Updateless decision theorists. I am not aware of an authoritative EDT treatment of this formulation of the problem.

The Merge Operator

We can formalise this notion of "putting on the blindfold" in my framework. Whenever we have a decision space  with some non-predicted observation  such that  we can propose the new decision space  where and . Note that only the observability of the problem is affected, everything else stays the same.

For example, applying this operator to the formalisation of Transparent Newcomb above yields exactly the formalisation of Newcomb's problem.

CDT Will Pay for Ignorance

While it is often accepted that paying for ignorance is a behaviour unique to EDT, it can be shown that in some problems, CDT also exhibits this behaviour.

Mechanical Newcomb

Mechanical Newcomb can also be described as "Causal Newcomb's Problem".

• Both boxes are made of a wire mesh and hence mostly transparent. The boxes are far enough apart that you can only reach one at a time.
• The large box contains $10 and the small box $1.
• Opening the large box permanently locks the small box, meaning that two-boxers must start with the small box.
• Opening the small box triggers the flamethrower to incinerate the contents of the large box.

Since this problem does not involve any kind of decision-sensitive prediction, all of CDT, EDT and UDT agree (that you should one-box).

Mechanical Newcomb by itself is (I think) largely uninteresting, but it can be used to build interesting problems.

Semi-Mechanical Newcomb

One week early, a highly-accurate predictor predicts what the agent would do if they were to play a game of Newcomb's Problem, then, a coin is flipped. If heads, the room is set up according to Newcomb's Problem and the prediction. If tails, the room is set up according to Mechanical Newcomb (no prediction needed).

Verdicts: All theories one-box in the mechanical half. UDT and EDT one-box in Newcomb's problem, CDT two-boxes there.

Semi-Mechanical Newcomb (Opaque)

The same problem, but with the lights turned off, meaning the agent can no longer see which sub-game they are playing.

 where  is Semi-Mechanical Newcomb from above.

Verdicts: All theories one-box.

While no agent knows which game they are playing, UDT and EDT know that they are definitely playing a game where they would one-box, so it is unsurprising that they one-box.

PS-CDT one-boxes regardless of its priors over the prediction, since one-boxing is $9 better in the mechanical branch, whereas two-boxing is only $1 better in the Newcomb branch.

For a similar reason, one-boxing is the only ratifiable action for NR-CDT. To see this, consider that if we fix the kind of agent you are and compare the actions available on that basis (as NR-CDT requires), you are comparing between a 50-50 split of being $1 better or $9 worse (two-boxing) and a 50-50 split of being $9 better or $1 worse (one-boxing).

If we compare expected earnings in the transparent and opaque versions of Semi-Mechanical-Newcomb, we find that both CDT variants score $5.55 and $9.95 respectively^[11], meaning both gain from ignorance, but would they pay for it? I cannot directly model this since I do not yet have definitions for multi-step problems, but I can sketch out how this could go.

In the first version, we have a single room with an external light switch. A coin is flipped, with heads leading to a prediction and Newcomb's problem, and tails to Mechanical Newcomb. A week later, the CDT agent shows up and is offered the opportunity to turn the light switch to "off" for $2. The agent knows that the predictor can accurately predict both whether they will flip the switch and which boxes they will open as a result.

PS-CDT does not pay, even though it would benefit by doing so. This is because it does not believe that it will benefit. PS-CDT will calculate the probability of the large box being full (conditional on the coin being heads) to be a certain value, which it will then hold fixed for the rest of its calculation. Regardless of its priors over the predictions, it is therefore guaranteed to conclude that keeping the lights on is $0.50 more valuable in expectation, since it allows for taking the extra dollar in the Newcomb-branch.

Fully-transparent Naive Ratificationist CDT correctly values the opaque game at $4.40 more valuable in expectation than the transparent game. However, it would not pay as the act of paying itself is, as best I can tell, not ratifiable.

Once the light-switch choice has been made, the only ratifiable options for box-opening are to one-box if the lights are off and to one-box/two-box if the lights are on in the Mechanical/Newcomb branches respectively. A fully-transparent agent would know this. Therefore an NR-CDT agent is either a) the kind of agent that will [lights on  one-box/two-box] or b) the kind of agent that will [lights off  one-box]. Any agent that knows itself to be a) will find b) more appealing since it knows that the large box in the Newcomb-branch will likely be full and b) would let it take an extra dollar there. Any agent that knows itself to be b) will not want to be a) as that would mean losing a dollar in the Newcomb-branch for no upside. Therefore turning the light off (even if free) is not ratifiable.

However, consider a slightly different setup of the problem. There are two rooms, one with the lights fixed on, and one with the lights fixed off. Only one of these rooms can be unlocked at any given time. The "light switch" controls which room is unlocked. A coin is flipped, with heads meaning both rooms are set up according to Newcomb's Problem adn tails meaning Mechanical Newcomb. For heads, each room has its own predictor that predicts what the agent would do if it found itself in that room (either by choice or being forced inside). Once again the switch is set to "on" (lit room unlocked) and can be switched to "off" for $2.

In principle, PS-CDT may choose to pay here, if its priors suggest that the predictor of the dark room is sufficiently more likely to have filled the large box in the Newcomb-branch. The version I have formalised would not pay as it believes all predictors to be uniform random.

NR-CDT would pay $2 to flip the switch, since it will be confident that (conditional on the coin landing heads) the lit room's big box almost certainly contains $0 and the dark room's box almost certainly contains $10. Since the light and dark rooms are now separate, knowing that you are the kind of agent who would pay $2 to flip the switch does not make refusing to flip the switch more appealing, since neither of the predictors care about the agent's switch-flipping decisions.

The implied conclusion here is that NR-CDT would not pay to refuse information as long as the refusal has no causal consequences as refusing would never be ratifiable. This requires more investigation to see what can actually be proved.  On the other hand, it seems certain that such agents would pay considerable amounts of utility in order to be allowed to play a strictly less visible version of a game, provided the less visible game is separate from the more visible game. For me, this is close enough to be able to say "CDT pays for ignorance". 

Updateful Theories will Pay for Ignorance

There is a sense in which the Fully-Transparent Epistemology is the perfectly-calibrated updateful epistemology. At every observation it provides only the expectations for that observation (is updateful) and calculates exactly the probabilities that the paired selection rule would encounter each of the start states (under each supposed action at ) (is perfectly calibrated).

At the predicted observation , we call an action  absurdly dominant by M if the expected value of actually performing  at  (for any supposed action informing the prediction) is higher than the value of any other action accessible at  (for any supposed action, including different actions) by at least M. A decision rule respects absurd dominance beyond M if there is some fixed M such that the rule will always select any action that is absurdly dominant by a margin of at least M.

 For any such decision rule (paired with Fully-Transparent Epistemology) we can, using the tricks like Mechanical Newcomb, produce a  such that the resulting Decision Procedure will (correctly) value some merge of  higher than , despite the merge being strictly less visible. This suggests that any reasonable perfectly-calibrated updateful Decision Procedure will pay for ignorance in at least some cases. Proving this properly would require multi-step problems amongst other extensions.

Formalised Problems

Many of the numbers here have been tweaked slightly to make them more compact to write down, but I do not believe I have significantly impacted the structure of any of these problems.

Counterfactual Mugging

In Counterfactual Mugging, a highly-accurate predictor flips a coin. If heads, they ask you for $1. If tails, they give you $10 if they predict that you would have paid if the coin had landed tails.

Verdicts

Conditioning on the observation  makes the value of  drop to -1 for all CDT and EDT variants (compared to  for ) so they all refuse to pay. UDT finds that   against   so UDT pays.

Counterfactual Mugging with Stick

This variant adjusts the "heads" branch of Counterfactual Mugging. In this version's branch, the predictor holds in one hand $10, and in the other hand a two-foot length of 2x4. The predictor says:

"Now I know what you're thinking, did they predict that I would give $1 on observing tails, or maybe only $0? In all the excitement you might have lost track yourself. But being as I am , the most powerful predictor in the world and can predict your counterfactual actions with 99% accuracy, you have to ask yourself one question, 'do I feel updateless?'"

"Well do ya, punk?"

You can choose to ask for the money or say nothing. If you say nothing, nothing happens. If you ask for the money and were predicted to pay in Tails, you get $10. If you ask for the money and were predicted not to pay in Tails, you get smacked around the head with the 2x4.

Verdicts

This problem illustrates the importance of having a transparent epistemology. Fully transparent variants of CDT or EDT know that they would not pay and decide not to risk the beating. Partially transparent or prediction-skeptic variants calculate that the risk is worth is and usually eat the 2x4. Updateless agents confidently ask for the $10 (knowing that this is what the overall best-performing agent would do) and usually get $10. 

XOR Blackmail

In XOR Blackmail, the agent lives in a village where every house has a 50% chance of being infested with termites. Termites will seriously damage houses (to the value of $10) if they infest them for long enough, but most people cannot detect them until it is too late.

In the village lives a highly-accurate predictor (who also happens to be an excellent detector of termites) who writes the following letter to all residents of the village.

Dear [Resident],

I have inspected your house for termites, and predicted whether or not you would send me $1 upon receiving this letter. If I predicted that you would pay, I sent you this letter if and only if your house does not have termites. If I predicted that you would not pay, I sent you this letter if and only if your house has termites.

Regards

Omega

Verdicts.

 A letter received by an agent-who-would-pay is nearly conclusive evidence against termites, and whereas it is nearly conclusive evidence for termites for an agent-who-would-not-pay. EDT therefore pays (conditional on receiving the letter, the expected utility of paying is much better than not paying). Paying is not ratifiable (given that you know you are the kind of agent who pays, not paying is preferable) but not paying is (if you are the kind of agent who doesn't pay and receives the letter, you already have termites), so NR-CDT doesn't pay. PS-CDT believes that either option leads to the same probability of termites, so doesn't pay to save $1. UDT observes that both paying and non-paying agents have termites with the same frequency, but paying agents sometimes lose $1 to the letter, so it doesn't pay.

Clone PD

This is a formalisation of a version of the Prisoner's Dilemma with two agents who are near-perfect psychological clones of each other. I model this with a decision-sensitive predictor who can puppet the clone to act in the same way that "the primary agent" is predicted to act. This version of the prisoner's dilemma involves the choice of pressing one of two buttons. The first button (labelled "Cooperate") sends $10 to your clone, and the second button (labelled "Defect") sends $1 to you.

Verdicts

Formalised in this way, Clone PD is structurally identical to the formalisation of Newcomb's Problem above, and the verdicts are equivalent (all CDT variants Defect, all EDT variants and UDT cooperate).

Death in Damascus

In Death in Damascus, the agent starts in Damascus knowing that they have an appointment with Death the next day. Death is a highly-accurate predictor who will predict whether the agent stays in Damascus or flees to Aleppo, and move to the corresponding city. For simplicity, in this version of the problem I assume that Death's presence directly causes the agent's death, and that Death picks people to play this game entirely at random.

Verdicts.

UDT, EDT and PS-CDT are all indifferent. UDT observes that fleeing and non-fleeing agents do equally poorly on average, EDT calculates that the expected value of fleeing or staying is equally poor, and PS-CDT believes that death is equally likely to be in either city. This is one of the canonical problems where PS-CDT's prior beliefs about the prediction matter, as it will typically avoid whichever city it thinks death is more likely to be in. If we assume a very slight utility cost to travelling to Aleppo, then all the above theories prefer to stay (apart from PS-CDT with moderate-or-stronger priors on Death being in Damascus).

NR-CDT fails to return a verdict here, as neither staying nor fleeing is ratifiable (if you know you are about to choose to stay or flee, you feel very strongly that you should switch). A cost to travelling to Aleppo does not help, as the penalty of meeting Death means the intention to stay is still not ratifiable. There are solutions to allow calibrated versions of CDT resolve this problem, but I do not address them here.

Bomb-Newcomb

Bomb-Newcomb originates from Will Macaskill as a proposed counter-example to updateless decision theories. In this problem the agent faces a tranparent box containing either a small amount of money or a bomb wired to explode when the box is opened.  A highly-accurate predictor requested money to be placed in the box if and only if they predicted that the agent would open the bomb-box if they saw it.