The first half of this post is a neutral explanation of EDT and CDT. The second half defends EDT and argues against CDT. If you want to learn properly about EDT, you should read Arif Ahmed's Evidence, Decision, and Causality (2014).

Table of contents

Motivating example: The Prisoners' Dilemma with a copy

Defining causal and evidential decision theory

Causal Decision Theory (CDT)

Evidential Decision Theory (EDT)

Further illustrating example: Newcomb's Problem

Adjudicating between EDT and CDT

EDT solves the Smoking Lesion scenario correctly

Extracting money out of CDT agents using the "Adversarial Offer"

Closing side remarks

Other arguments

EDT and CDT don't self-modify to convergence

On Functional Decision Theory

Motivating example: The Prisoners' Dilemma with a copy

Imagine you find yourself in the following situation: You and your opponent each are given the choice to either earn $1 or to earn the other player $3. Earning the other player $3 nets you $0. Both of you are asked separately, only once, don't care about each other, and there are also no reputational effects. This problem is called the Prisoners' Dilemma.

This problem seems easy enough. Earning $1 is better than earning $0, so if you don't care about the other person, you should clearly just take the $1.

But what if we introduce a twist: Your opponent is actually a copy of you that some dastardly scientist created. If you're an AI, the other player might just be another instance of you. It seems very likely that whichever choice you make, your opponent will make the same choice! Your actions are correlated with each other.

You would be better off if you both cooperated. Yet, if the other person cooperates, you would be better off if you defect and if the other person defects, you are also better off defecting! How should you decide in this situation?

Defining causal and evidential decision theory

Academics have come up with competing theories with the two most prominent ones giving opposite recommendations. Causal decision theory (CDT) recommends you defect. Evidential decision theory (EDT) recommends you cooperate.

Both causal and evidential decision theory have expected utility maximization as their core: They each say that you should make your decision in three steps: Associate with each action a a probability distribution over outcomes . Then compute expected utilities by multiplying the probabilities with ...

Intro to acausal interactions

Intro to acausal interactions