One thing that I've noticed when talking to people about decision theory is that there is a lot of confusion about what an evidential decision theory agent actually is. People have heard that it doesn't smoke in Smoking Lesion and that it pays in X-Or Blackmail, but that is what it does. They may know that it doesn't do Pearlean Graph surgery or differentiate correlation from causation in some sense, but that is what it is not. They may even know it calculates an expected value using the probability distribution P(O|S&A), but that is just a mathematical formalisation which anyone can quote without any real understanding. I've taken a stab at clarifying it in a few short-form posts, but people didn't seem to find them very enlightening.

Even now, my understanding is still weaker than I'd like. Like I just spent over fifteen minutes thinking about whether it would be accurate to characterise it as an agent that is purely concerned with correlation with no notion of causation. I thought this would be accurate at first, but then I realised that under reasonable assumptions, an EDT agent wouldn't expect buying a diamond necklace to increase its wealth. After all, it would be able to notice that an increase in wealth tends to precede buying such a necklace, but this pattern doesn't occur in reverse. In other words, it tends to have at least some ability to model causation.

Anyway, it seems, to me at least, that it would be rather useful for someone to have a go at providing a clear explanation of what exactly it is.

New to LessWrong?

New Answer
New Comment

1 Answers sorted by

abramdemski

Jan 02, 2020

60

I'm posting a short response rather than there be none, although I think you are calling for a longer more thoughtful response.

I would simply say an evidential agent selects an action via ; that is, it evaluates each action by (Bayes-)conditioning on that action, and checking expected utility.

Of course this simple formula can take on many complications when EDT is being described in more fleshed-out mathematical settings. Perhaps this is where part of the confusion comes from. There is some intuitive aspect to judging whether a more complicated formula is "essentially EDT". (For example, the classic rigorous formulation of EDT is the Jeffrey-Bolker axioms, which at a glance look nothing like the formula.)

But I would say that most of the issue you're describing in the OP is that people think of EDT in terms of what it does or doesn't do, rather than in terms of this simple formula. That seems to be genuinely solved by just writing out when people seem unclear on what EDT is.

Also, note, the claim that EDT doesn't smoke in smoking lesion is quite controversial (the famous tickle defense argues to the contrary). This is related to your observation that EDT will often correctly navigate causality, because the causal structure is already encoded in the conditional probability. So that's part of why it's critical to think of EDT as the formula, rather than as what it supposedly does or doesn't do.