jackmastermind

A twin guard-inmate dilemma (twin GID) is an asymmetric game that breaks FDT. [Image: GPT Image-1] 0. Introduction TL;DR: FDT and UDT diverge in how they handle "behave as you would have ideally precommitted to behaving" in asymmetric games where a player is assigned a role after a deterministic clone is made. FDT updates, whereas UDT does not. ∴ an agent who knows in advance that they will enter one of these games would convert to UDT, not FDT, on this problem. [UPDATE: this applies to the formulation of FDT in the paper, but not necessarily to Yudkowsky & Soares' "preferred" version of FDT; see Menotim's comment] I wrote a version of this post on my substack; it was for a less technical audience, and at the time I didn't understand updateless decision theory. I assumed that UDT and FDT just used different methods to compute the same recommendations. I was wrong! In fact, there are very simple scenarios in which FDT does not recommend precommitting to itself. 1. Definitions According to Yudkowsky & Soares' "Functional Decision Theory: A New Theory of Instrumental Rationality," FDT, CDT, and EDT all maximize expected utility as defined by this formula: EU(a):=N∑j=1P(a↪oj;x)⋅U(oj) > where o1,o2,o3 . . . are the possible outcomes from some countable set O; a is an action from some finite set A; x is an observation history from some countable set X ; P(a↪oj;x) is the probability that oj will obtain in the hypothetical scenario where the action a is executed after receiving observations x; and U is a real-valued utility function bounded in such a way that [the above equation] is always finite. > > … > > From this perspective, the three decision theories differ only in two ways: how they prescribe representing [the world-model] M, and how they prescribe constructing hypotheticals Ma↪ from M. (emphasis mine) From here, the three decision theories are formalized by: EDT(P,x):=argmaxa∈AE(V|Obs = x,Act = a)CDT(P,G,x):=argmaxa∈AE(V|do(Act = a),Obs = x)FDT(P,G,x):=