This is a linkpost for https://glizen.com/radfordneal/newcomb.abstract.html
I'm not an FDT advocate (main reason being: FDT is not well-defined) but I can try saying what FDT* would do. For context, FDT* is defined in the FDT paper as an updateless, logical form of EDT. (The paper itself uses a formulation that uses 'true'. This is confusing, I'd rather talk about the version that just conditions on a logical fact. I'll pretend FDT* is just conditioning on logical sentences instead of using 'true'.)
Start with smoking lesion. Let
I'm guessing FDT* is going to smoke here at least for tickle defense reasons. Since when we already condition on desire to smoke, we get ~no evidence about cancer based on the action of smoking. Conditioning on the logical fact informs us about the action, but doesn't seem to additionally inform about the cancer.
Next, let's consider the case where the gene that causes cancer also has effects on the decision rule. Again, let
Let's suppose NSDT is some decision theory that doesn't smoke in this sort of situation, and CDT does smoke. (Perhaps NSDT is like a sloppy heuristic form of EDT.) It seems one way to get these population statistics is for genes to affect relative adoption of CDT or NSDT. There are also the (presently unknown) statistics for FDT*, and genes can affect FDT* prevalence.
We need to decide whether the FDT* agents know about population statistics for FDT* agents. First we consider the case where they don't. Maybe FDT* is very rare in the general population, so statistics don't imply much about FDT* agents.
Case 1: "gene causes both cancer and formation of brain structures which implement FDT*". A priori, is it good news if FDT* agents smoke when observing this? Yes; they'd have cancer and FDT* either way, the only remaining evidential consequence is whether they smoke.
Case 2: "gene causes cancer and also causes a generic tendency to adopt some decision theory which endorses smoking in this case 2". A priori, is it good news if FDT* agents smoke when observing this? Basically, if this is the case, then more of the people with the gene will adopt FDT*. This increases the proportion of FDT* agents who have cancer. Conversely, if FDT* agents smoke when observing this scenario, then people without the cancer gene will avoid adopting FDT*. So we're comparing situations where "people with the cancer gene often adopt FDT*" and where "people without the cancer gene often adopt FDT*". This gets complicated and seems pretty dependent on people's utility functions. If these people are utilitarian-ish, they might prefer it to be the case that people without the cancer gene tend to adopt FDT*, because that makes FDT* utilitarians healthier than the alternative scenario. In that case, FDT* would recommend not smoking.
It gets more complicated when some of the FDT* agents get to see what the other FDT* agents decide to do (and whether they get cancer) in the population statistics. Roughly it seems they should be making the same decisions as the earlier agents in the usual case, although I'm not totally sure.
Ok, let's move on to impersonal Newcomb. The basic case here is: "A priori, it would be great if FDT* one-boxed. That way, the predictor's samples would be of FDT* agents one-boxing, so the predictor would put money in box A for FDT* agents, such as me." So FDT* one-boxes. (I'm assuming, here, that the population involves a significant portion, >10%, of FDT* agents.)
For Newcomb's problem based on categories, assuming categories are at least as fine as "which decision theory is the person using", it seems FDT* is similarly going to one-box. This is because, a priori, you don't know your exact observations; it's good news to learn "FDT* agents observing their category is C and their auxiliary observations are O decide to one-box" since that's generically evidence that FDT* agents with category C will be rewarded, and it's plausible a priori that "you" will have category C.
That wraps up summarizing / guessing what FDT* does. A few other comments. First, I don't have much of an opinion on what FDT does, since the paper doesn't define it very well / unambiguously. Second, I appreciate the observation that a CDT computational functionalist will one-box in a variant of Newcomb's problem where they are actually simulated. It overlaps with general analysis of CDT in imperfect recall games (1, 2, 3). It does seem like the best case for one-boxing is precisely the cases where the agent is actually instantiated, such that the agent should consider "I am in a simulation" plausible a posteriori. Third, it looks like the "gene affects decision theory adoption" is the most ambiguously specified of these four scenarios, so it might be worth examining specific variations.
Hi. Thanks for your detailed reply.
Regarding smoking lesion scenarios, my thoughts (see the Discussion section) were that both when the lesion affects desire and when it affect the decision rule used, FDT would recommend smoking, since imagining that your decision function outputs "smoke" or "don't smoke" affects your utility only through its affect on whether you obtain the pleasure of smoking - it's assumed that you have no concern for other people's pleasure.
So on this view, it shouldn't matter whether the lesion affects the decision rule used in the sense of inclining people to use FDT, or inclining them to use CDT, or inclining them to use some other decision rule that results in them smoking, or some combination of these. I assume that all you know is the correlatio between decision (however made) and cancer, but I don't think it would matter if you had more detailed information.
If you have altruistic motivations, it's a different problem, which also seems interesting, and in which the proportion of smokers who use FDT starts to seem relevant, so the action FDT recommends would depend on beliefs about that, and also (as you note) on any information about cancer in FDT agents in particular.
As you say, the proportion of past Participants (on whose actions the Predictor based their prediction) who used FDT could be relevant for the impersonal Newcomb problem too. This seems a bit problematical, since presumably one wants to count as "FDT agents" people who have never heard of FDT, but are consciously or unconsciously behaving as an FDT agent.
The arguments seem similar (though more complicated) in a non-fantastical standard Newcomb problem with prediction using categories. It seems so extravagent to me, though, to be imagining that so many people decided one way or another as you decide - if you decide to one-box, you're imagining that you get $1M, but also that all those other people got $1M, which could affect the general economy, causing inflation and other bad things, so maybe two-box instead? It seems too much...
Here's one way to analyze the selfish case, in a case where the gene affects decision theory adoption. Alice adopts a utility function that is like "Alice has a good life"; this contains some sort of specification of Alice. Alice uses FDT*. Alice reasons about what would happen if FDT* does or doesn't recommend smoking. Conditional on FDT* recommending smoking, it is a priori the case that people with the gene adopt FDT*. Conditional on FDT* recommending not smoking, it is a priori the case that people without the gene adopt FDT*.
It might depend on how "Alice" as in "Alice has a good life" is defined. Assume "Alice" is not defined genetically. In which case, a priori, it might or might not be the case that "Alice" has the gene. So within each of the above scenarios (FDT* recommends smoking or not) we also need to split on whether Alice has the gene. This leads to a 2 x 2.
So FDT* recommends smoking. It's a bit complicated though, because there could be downstream effects of decision theory adoption other than smoking or cancer.
The arguments seem similar (though more complicated) in a non-fantastical standard Newcomb problem with prediction using categories. It seems so extravagent to me, though, to be imagining that so many people decided one way or another as you decide - if you decide to one-box, you're imagining that you get $1M, but also that all those other people got $1M, which could affect the general economy, causing inflation and other bad things, so maybe two-box instead? It seems too much...
I don't think we should take inflation into account. Reduce number of people or payouts till it's not an issue, to simplify problem. It would be ridiculous to offer Newcomb to over 1 billion people.
I still think FDT* one-boxes in Newcomb and it's different from the smoking lesion scenario. Basically we imagine Alice asks, a priori, is it good news if FDT* one-boxes. With the smoking lesion case, we know the genes were assigned randomly, so we can analyze the 4 cases above while assuming P(Alice has cancer | FDT* recommends smoking) = P(Alice has cancer | FDT* recommends not smoking). However this is pretty different in the Newcomb scenario.
Conditional on FDT* recommends one-boxing, the predictor will fill the box with $1M for someone who fits the category of a FDT* agent. Conditional on FDT* recommends two-boxing, the predictor will not fill the box with $1M for someone who fits the category of a FDT* agent.
So I think FDT* recommends one-boxing.
This is a link post for my paper on When to Take One Box, When to Take Two: A Concrete Analysis of Newcomb's Problem (also on arxiv and philsci-archive).
I assume you all know about Newcomb's problem - it's a mainstay of lesswrong discourse. And you probably also think that taking one box is obviously right, on the grounds that Rational agents should WIN, which is known as the `if you're so smart, why ain'cha rich?' argument in the philosophy literature.
But what you think you know ain't always true. And you're more likely to make mistakes when reasoning about abstract or fantastic scenarios, than in concrete situations that can actually occur. In my paper, I argue that a concrete, actually-realizable version of Newcomb's problem is possible, with the Predictor having substantial, but not near-perfect, accuracy. I also argue that in this situation, you should take two boxes.
If instead the Predictor simulates your thoughts, via some fantastic method (that may well be impossible), then you should take one box.
I focus on what Evidential Decision Theory (EDT) and Causal Decision Theory (CDT) recommend in these scenarios, concluding that CDT gives the right answers, and only discuss Functional Decision Theory (FDT) briefly at the end. Also, I explicitly assume that the decision maker is human, whereas many on lesswrong have a tendency to immediately shift to considering decisions made by an AI. So adherents of FDT on lesswrong might think my discussion is irrelevant to their interests. But I think they should still read it. If FDT doesn't work in concrete situations faced by humans, then you'd want to know that, right?
I introduce a progression of problems in order to carry causal intuitions from common situations on to Newcomb's problem. These are:
CDT recommends the analogue of 'two boxing' for all these problems. EDT recommends `two boxing' for the first, and 'one-boxing' for the rest. What does FDT recommend? I'm not sure, but perhaps it recommends 'two-boxing' for the first two and 'one-boxing' for the last two. But if so, is the change going from (2) to (3) justifiable? Or maybe FDT recommends `two-boxing' for (3) and/or (4). But then, the argument that FDT gives you more money no longer works. I'd be interested in hearing what FDT advocates think about this.