I've been trying to gather my thoughts for my next tiling theorem (agenda write-up here; first paper; second paper; recent project update). I have a lot of ideas for how to improve upon my work so far, and trying to narrow them down to an achievable next step has been difficult. However, my mind keeps returning to specific friends who are not yet convinced of Updateless Decision Theory (UDT).

I am not out to argue that UDT is the perfect decision theory; see eg here and here. However, I strongly believe that those who don't see the appeal of UDT are missing something. My plan for the present essay is not to simply argue for UDT, but it is close to that: I'll give my pro-UDT arguments very carefully, so as to argue against naively updateful theories (CDT and EDT) while leaving room for some forms of updatefulness.

The ideas here are primarily inspired by Decisions are for making bad outcomes inconsistent; I think the discission there has the seeds of a powerful argument.

My motivation for working on these ideas goes through AI Safety, but all the arguments in this particular essay will be from a purely love-of-knowledge perspective.

There are a few different ways we can think about the goals of Decision Theory as an academic field. I'll highlight three:

Advice Stance: Decision theory is the study of good advice, divorced from any particular subject matter -- just the pure theory of what sort of advice is best. This is a theory of what advice should be persuasive, not what advice is persuasive. Subjects are imagined to face some choice (a decision problem), and the advice-giver tells them how to proceed. The decision theorist who can provide the best advice for the broadest variety of decision problems 'wins' (their views are adopted by other decision theorists). In some ways this focuses on decision theory as a human activity; trying to construct an ideal advice-giver.
Design Stance: Decision theory is about designing decision procedures which choose well. These decision procedures face off in a variety of decision problems, and again, the one which can provide the best advice for the broadest variety of decision problems wins. The design stance is correlated with thinking about artificial intelligence; if we completely control the design of an agent, what's the best thing to build?
Naturalist Stance: Decision theory is a theory built in response to observing an emp

...

Limitless

Limitless