I've been genuinely confused about all these antropics stuff and read your sequence in hope for some answers. Now I understand better what are SSA and SIA. Yet am not even closer to understanding why would anyone take these theories seriously. They often don't converge to normality, they depend on weird a priori reasoning which doesn't resemble the way cognition engines produce accurate maps of the territory.
SSA and SIA work only in those cases where their base asumptions are true. And in different circumstances and formulations of mind experiments different assumptions would be true. Then why, or why, for the sake of rationality are we expecting to have universal theory/referent class for every possible case? Why do we face this false dilemma, which ludicrous bullet to bite? Can we rather not?
Here is a naive idea for a supperior antropics theory. We update on antropic evidience only if both SSA and SIA agree that we are to. That saves us from all the presumptious cases. That prevents us from having precognitive, telekenetic and any other psychic powers to blackmail reality, while allowing us to update on God's equal numbers coin toss scenarios.
I'm pretty sure there are better approaches, I've heard of lots of good stuff about UDT, but haven't yet dived deep enough inside it. I've found some intuitively compelling aproaches to antropics on LW. Than why do we even consider SSA or SIA? Why are people amused by Grabby Aliens or Doomsday Argument in 2021?
I skimmed the sequence looking for "How come I can move without bumping into a copy of myself?".
Rephrasing: When I close my eyes and set aside which observer I am, the SIA prior places me in a densely populated environment, and then I open my eyes and find that my timeline doesn't look outcome pumped for population density.
In fact, we find ourselves on the spawning planet of our civilization. Since the Fermi paradox is dissolved, a randomly selected observer should have been much later; instead we find ourselves near the fulcrum of the plot. Therefore we should update towards UDT: The anthropic assumption that we are more likely to find ourselves in the positions that matter. Oh look, I accidentally solved the simulation hypothesis and Boltzmann brains and the dreaming butterfly. Curious, this even predicted that we find ourselves with above-average intelligence and working on AI math.
One may also confuse the density of observers and the numbers of observers. You are more likely to be in the region with the highest number of observers, but not of the highest density. A region can win the biggest number of observers not because it has higher density, but because it is larger in size.
For example, Tokyo is densiest city, but most people live in rural India.
SIA can be made to deal in densities, as one must with infinities involved.
Though absolute SIA also favors the hypothesis that I find myself in a post-singularity megacivilization, to the point that our observations rule SIA out.
You are most likely in the singularity post-civilization. But in simulation which it created. So no SIA-refutation here.
SIA can be made to deal in densities, as one must with infinities involved.
I didn't get what do you mean here.
I see - SIA can be finagled to produce the "we find ourselves at history's pivot" we observe, by rationalizing that apparently something somewhere is desperate enough to accurately predict what the people there would do to make most of all the anthropic mass be there. I admit this has a simplicity and ring of history to it.
re densities: If two universes are infinite and both have infinite observers and they only differ in whether every observer sees an observable universe with 10^10 other observers or 10^20 other observers then we could, if we wanted, call one more likely to find oneself in than the other.
Yes, indeed, "measure monsters" could fight to get biggest share of measure over desirable observers thus effectively controlling them. Here I assume that "share of measure" is equal to the probability of finding oneself in that share under SIA. An example of such "measure monster" may be Friendly AI which want to prevent most people to be in hands of Evil AI, so it creates as much copies of people as it can.
Alternatively, very strong and universal Great Filter Doomsday argument is true, and Earth is the biggest possible concentration of observers in the universe and will go extinct soon. Larger civilizations are extremely rare.
But I think that you want to say that SIA prediction that we are already in "measure monster" is false, as we should observe much more observers, maybe a whole Galaxy densily packed with them.
Your last paragraph is what I meant by "find myself in a post-singularity megacivilization".
Your first paragraph misunderstands my "SIA can be finagled". The ring of history comes not from "AIs deliberately place a lot of measure on people to compel them", but from "AIs incidentally place a lot of measure on people in the process of predicting them". Predicting what we would do is very important in order to correctly estimate the probabilities that any particular AI wins the future, which is a natural Schelling point to set the bargaining power of each acausal trader.
Agree that AI will do a lot past simulations to predict possible variants of world history and even to try to solve Fermi paradox and-or predict behaviour of alien AIs. But it could be outweighed by FAI which tries to get most measure in its hands, for example to cure past sufferings via indexical uncertainty for any possible mind.
I am pretty sure that UDT doesn't say we should expect to be important, but maybe you should elaborate just in case :P
Infinite copies of you may each spend 1 point. If #1 does, everyone gains 2 points. Equipped with the Self-Importance Assumption that we are >50% likely to be #1, CDT acts like UDT.
Suppose we modify the game so that you can now spend 1 point on the "if you're #1, everyone gets 2 points" gamble, and also can choose to spend 1 point on the "if you're #1, you get 2 points" gamble. UDT still is fine, self-important CDT loses all its gains. I feel like the moral of the story is "just don't be CDT."
You're right! My own search for counterarguments only came as far as "as the amount of points everyone gains decreases to 1, the probability of #1 required to match UDT rises to 1" - I didn't manage to leave the parameter space I'd constructed, to prove myself wrong.
And yet. Yes, for any probabilities, CDT will take the one action iff it takes the other, and UDT has some probability distribution (used to weight each copy's utility) such that it takes one action but not the other. Does every game have a probability distribution where CDT and UDT agree? Can we naturally construct a sane such anthropic assumption? The utility function isn't up for grabs, but this'd still seems like a hint.
Awesome sequence!
I wish that discussions of anthropics were clearer about metaphysical commitments around personal identity and possibility. I appreciated your discussions of this, e.g., in Section XV. I agree with what you, though, that it is quite unclear what justifies the picture “I am sampled from the set of all possible people-in-my-epistemic situation (weighted by probability of existence)”. I take it the view of personal identity at work here is something like “‘I’ am just a sequence of experiences S”, and so I know I am one of the sequences of experiences consistent with my current epistemic situation E. But the straightforward Bayesian way of thinking about this would seem to be: “I am sampled from all of the sequences of experiences S consistent with E, in the actual world”.
(Compare with: I draw a ball from an urn, which either contains (A) 10 balls or (B) 100 balls, 50% chance each. I don’t say “I am indifferent between the 110 possible balls I could’ve drawn, and therefore it’s 10:1 that this ball came from (B).” I say that with 50%, ball came from (A) and with 50% the ball came from (B). Of course, there may be some principled difference between this and how you want to think about anthropics, but I don’t see what it is yet.)
This is just minimum reference class SSA, which you reject because of its verdict in God’s Coin Toss with Equal Numbers. I agree that this result is counterintuitive. But I think it becomes much more acceptable if (1) we get clear about the notion of personal identity at work and (2) we try to stick with standard Bayesianism. mrcSSA also avoids many of the apparent problems you list for SSA. Overall I think mrcSSA's answer to God's Coin Toss with Equal Numbers is a good candidate for a "good bullet" =).
(Cf. Builes (2020), part 2, who argues that if you have a deflationary view of personal identity, you should use (something that looks extensionally equivalent to) mrcSSA.)
That said, I can think of an experiment that at least doesn't throw a compilation error. On a quantum computer, simulate different people in different timelines. If onlookers paying different amounts of attention to different of those timelines pumps amplitude into those timelines, and I see no reason it should, this is straightforwardly measurable. People aren't ontologically basic, so the same pumping should already warp the lesser calculations we do today.
When people look for excuses to believe phi they will be more likely to find excuses to believe phi than excuses to believe not-phi.
The thing that's important in a prediction model is that it is accurate. Therefore, no interventions. The protagonist is allowed to believe that he is the protagonist because that's what he believed in real life.
I'm not talking about the precision of our instruments, I say that you're measuring entirely the wrong thing. Suppose a box contains a cat and air. What has more probability, the cat or the air? That question makes no sense. Probabilities are about different ways the contents of the box might be configured.
I didn't talk about the quality of the excuse, I was remarking on why you were looking for it; and people are more likely to find what they're looking for than the opposite. But on the object level: Do you mean that onlookers would intervene within their simulation to keep the protagonist safe? That sounds like a rather bad strategy for modeling important events correctly. But you probably mean that this particular fact about the future might be learnable by direct observation in the past... I regret to inform you that quantum physics does not place different amplitude on different objects on the same planet, amplitude is a property of world states. The proposed soul quantity has nothing to do with physics, it is a high-level fact about minds and what they're modelling. It is a quantity of a kind with "how much do I care about each human?" and "how likely do I think each future to be?".
Sounds like you've written the bottom line of wanting to prove your status to others before you found the excuse of it being important to people to know not to mess with you.
We should not give more status to those who determine the fate of the world because then it will be determined not by those that seek to optimize it but those that seek status.
To measure soul you must pick some anthropic definition. If it's "who is most desperately-for-precision modeled by onlookers?", to measure it you acquire omnipotence, model all the onlookers, and check who they look at.
Throw a fair coin a billion times. If you find yourself in the timeline where it comes up heads every time, that's evidence that it has more soul than the others.
Then I would not put them in an Earth environment, and would not straddle them with akrasia. This bias sounds more caused by focusing on the humans that end up determining the fate of the cosmos for purposes of prediction.
In "Sleeping Beauty and Self-Location: A Hybrid Model", Bostrom comes up with a way to have Beauty believe that P(Tails) = 1/2, both before and after learning that it's Monday. The idea is to treat the situation as five separate possible cases: three cases before learning it's Monday (Heads^Mon_1, Tails^Mon_1, Tails^Tue_1) and then two cases after learning it's Monday (Heads^Mon_2, Tails^Mon_2). Once Beauty learns that it's Monday she's now in a new reference class, and since each world contributes exactly one observer moment, she ends up back at P(Tails) = 1/2. Here, "learning" doesn't mean narrowing the set of possible cases we conditionalize upon, as in standard Bayesian conditionalization, but dropping the model and replacing it with a new one having different measures. Note that there's no situation in which she'd be unaware of whether she's in the before or after learning cases, since she'd always know whether she has or has not learned that today is Monday, so treating them as being in the same probability space or not makes no difference.
This move is similar to the introduction of reference classes: a new variable is injected into the equation whose value depends on philosophical interpretation rather than well defined rules, and this variable fully determines the probability of each possible world. For instance, including her Heads^Tue (aspeep) state in the reference class makes her a thirder, while excluding it makes her a halfer.
So on the one hand we can fully determine the probabilities of each possible world by playing with reference classes, and on the other, we can decide whether to use conditionalization or instead replace the model with a new one by deciding when to embed learning steps into the model. Admitting these devices in probability theory would end up rendering the whole thing pretty meaningless insofar as it is a system of rules.
(Cross-posted from Hands and Cities. Previously in sequence: Part 1: Learning from the fact that you exist; Part 2: Telekinesis, reference classes, and other scandals; Part 3: An aside on betting in anthropics.)
This post is the last in a four-part sequence, explaining why I think that one prominent approach to anthropic reasoning (the “Self-Indication Assumption” or “SIA”) is better than another (the “Self-Sampling Assumption” or “SSA”). This part discusses prominent objections to SIA. In particular:
That said, even if SSA is worse than SIA, it’s not like SIA is sitting pretty (I especially don’t like how it breaks in infinite cases, and there are presumably many other objections I’m not considering). I briefly discuss whether we should expect to find a better alternative — the “Anthropic Theory X” above. My current answer is: maybe (and maybe it’s already out there), but Anthropic Theory X should probably keep SIA’s good implications (like “thirding” in Sleeping Beauty). And the good implications seem closely tied to (some of) the bad.
I close by quickly mentioning some of SIA’s possible implications in the real world (for example, re: doomsday arguments). I think we should tread carefully, here, but stay curious.
A lot of this sequence so far has been about objecting to SSA. So let’s return, now, to SIA’s problems: and in particular, the Presumptuous Philosopher.
I granted, earlier, that the Presumptuous Philosopher is a bad result (I’ll also discuss some more extreme versions of this result – for example, in infinite cases – in the next section). It seems strange to think that we should upweight scientific hypotheses in proportion to the number of people-in-our-epistemic-situation they posit or imply. In particular, people in your epistemic situation seem so … cheap. You can just 2x them, a trillion-trillion-x them, with a brief waggle of the tongue. Must our credences swing, wildly, in proportion to such whimsy?
I’d like to think that on the actually-good theory of anthropics, combined with our knowledge of everything else, this doesn’t happen so easily in the real world (but that it probably does happen in God’s coin toss type cases, as e.g. thirding in Sleeping Beauty seems to require). But I don’t have the actually-good theory of anthropics. Rather, what I have, so far, is SIA and SSA: and SSA, as I’ve discussed, seems to me pretty bad. What, then, should we do? And in particular, if SIA and SSA are our only options (they aren’t, but see section XVI for more), how can we live with ourselves?
I find it helpful, in such an unpleasant dialectical situation, to bring to mind the concept of a “good bullet,” which I will illustrate with an analogy to population ethics.
Once upon a time, there was a conclusion called the repugnant conclusion. This conclusion said that if you add zillions of only-a-bit-good lives in a population, then with only a brief waggle of the tongue, the goodness of that population can swing wildly – enough, indeed, to outweigh anything else. Such a conclusion was implied by a very natural, simple, intuitive theory of the goodness of populations (e.g., add up the goodness of the lives involved); it was supported by a number of extremely strong abstract arguments, based on seemingly undeniable principles; and the most immediate alternative theories immediately faced much worse counterexamples to boot. Indeed, there were even proofs that if you were going to deny the repugnant conclusion (or indeed, a weakened version of it), you would necessarily say something else very counterintuitive.
Few philosophers were actively excited about the repugnant conclusion. But some would stop at nothing to avoid it. Increasingly desperate, this latter group started saying crazy things left and right. They started putting people in hell to avoid making extra happy people who aren’t happy enough. They started denying that betterness is a consistent ordering. They started talking a lot about indeterminacy and incomparability and incommensurability. They started talking about how maybe ethics is impossible, maybe it’s no use making choices, maybe caring about stuff in a coherent way is a hopeless endeavor. Some went to their very graves, after decades, still writing papers on the topic, still searching for some way out. They had picked, it seemed, a bullet they would never bite, an immovable rock on which all else must be built, and thus, around which all else must contort — how garishly, whatever the costs.
To others, though, the situation was different. They felt like: “wait, so if I bite this one bullet that maybe isn’t even that bad from a certain perspective, and which seems like a natural and direct extension of reasoning I accept in other circumstances, then I get in return a nice simple intuitive theory, rather than one of these much-worse alternatives? This is sounding like a pretty good deal.” On this view, it’s not that the bullet, when you bite it, tastes good: it doesn’t. But if you bite it — or at least, if you allow yourself to consider the possibility of biting it — you’re suddenly allowed to see clearly again. Things come back into focus. The craziness is over. You can rest. (At least, for now.) In this sense, even if the bullet’s taste is bitter — even if you remain open to it being wrong, and interested in genuinely-more-attractive alternatives — it’s a “good bullet” nonetheless.
I’m not going to take a stand, here, about whether this “good bullet” conception of the repugnant conclusion is right. But I want to flag the possibility that we should be telling a similar story about the Presumptuous Philosopher.
One reason to suspect this is that Bostrom seems to treat the Presumptuous Philosopher as basically the end-of-discussion-objection to SIA, and my sense is that others do as well. Thus, the dialectic goes, “Of course, we could solve this ridiculous problem if we just used SIA. But: the Presumptuous Philosopher. Ok, back to SSA.” (OK, this is a bit of a caricature. But I do get a “this is the central and canonical counter-example to SIA” vibe from the Bostromian literature.) And you’ve been dismissing an otherwise-attractive theory centrally in virtue of a single case…
Another reason to suspect that the Presumptuous Philosopher is a good bullet is that when we try to avoid it, at least via SSA (and see section XVI for more pessimism), we get all the crazy contortions, epicycles, boulder-swervings, reference-class indeterminacies, solipsisms, metaphysical mistakes, and so forth discussed above. I do feel some pattern matching to the population ethics literature, here.
A final reason to suspect that the Presumptuous Philosopher is a good bullet that it’s basically just a restatement of the verdicts that we (or at least, I) want from SIA (e.g., thirding in Sleeping Beauty), and it follows from pretty straightforward and compelling arguments, like the Dorr-Arntzenius argument above (e.g., you should be 1/4 on everything when you’re woken up twice regardless; so what happens when you learn it’s not Heads-Tuesday?). I discuss this a bit more in section XVI below.
Indeed, to stretch the analogy with the repugnant conclusion further, I think there are actually some close spiritual similarities between totalism in population ethics and SIA in anthropics (thanks to Katja Grace for suggesting this). In particular, both of them — at least seen in a certain light — involve a central focus on possible people. That is, totalism cares not just about the people who actually live, but the people who don’t get live because you didn’t create them. Similarly, SIA cares, not just about the fact that you’re actually alive, but the fact that you might not have been: it treats you as a possible person who happened to become actual, rather than as someone who would’ve been actual no matter what. And one lesson the totalist takes away from population ethics is that possible people tend to get short shrift in actual people’s intuitive consideration. Perhaps something similar is happening with resistance towards SIA.
Even if the Presumptuous Philosopher is a strong candidate for a good bullet, though, this isn’t to say that we should just go ahead and bite. Heck, I’ve only looked at two theories, here, and only a smattering of considerations. But I think we should keep the possibility of biting on the table.
Let’s look briefly at a few other objections to SIA, including some more extreme variants on the Presumptuous Philosopher: namely, a “pascal’s mugging” variant, and an infinite variant.
The pascal’s mugging variant is just: I can posit extra people-in-your-epistemic-situation faster than you can decrease your credence on my hypotheses. Thus, boom, I suggest that the hypothesis that there are a Graham’s number of you-copies-having-your-experiences buried deep in your closet; I suggest that this world overlaps with a Graham’s number of other hidden realms containing people like us (see Olum 2000, p. 15); I suggest a Graham’s number of hypotheses like these; and so on. Isn’t this a problem?
Yeah, it does seem like a problem. But on the other hand, it feels like the type of “big number, not-small-enough probability” problem we (or at least, some of us) are kind of used to not having a particularly clear picture of how to solve (see here for some of the terrible trade-offs; also this); but which we don’t currently give up basic and plausible commitments in the face of. That is, you can make up ridiculously large numbers of lives-to-be-saved, too; but this doesn’t mean I stop viewing lives as worth saving (though I do get curious about what’s going on), or that I start thinking that I need to know how many lives have been saved already in order to decide whether to save another. I’m inclined to treat the possibility of making up ridiculously large numbers of observers-like-you in a similar way: that is, to worry about it, but not to freak out just yet.
Ok: but what about infinite cases? Doesn’t SIA become certain that the universe is infinite — and in particular, that it’s filled with infinitely many observers-like-us? And isn’t this obviously overconfident? Surely the universe, as it were, could be finite — what with finitude being an actual on-the-table scientific hypothesis, for example (see Sean Carroll’s comment at 13:01 here). And even if the scientists end up leaning hard towards an infinite universe, couldn’t it have been the case that it was finite instead? Isn’t that just: a way things could’ve been? And if there would’ve been observers in such a situation, wouldn’t SIA doom them to being infinitely wrong?
(Though note, of course, that you shouldn’t actually be certain of SIA, and so shouldn’t be certain of its conclusions. And it’s not like SSA is sitting pretty with respect to “no-certainty about infinites” type considerations: to the contrary, SSA becomes certain that we’re not in an infinite world, once it narrows down “who it is in that world” to any finite population (see Grace’s discussion here)).
But SIA’s infinity problems get worse. In particular, once it has become certain that it’s in some infinite world or other, it’s not actually particularly sure about how to reason about which. Suppose, for example, that God has a button that will create an infinite number of copies of you. If heads, he presses it once. If tails, he presses it twice. Here you are post coin-flip. OK, SIA, are you a halfer now? I guess so? No anthropic update for any worlds with infinitely many copies of you? But don’t we still want to be thirders in various cases like that — for example, in the actual world, if it’s infinite? What if we start using different sizes of infinity?
Yeah, look, I don’t like this either. In particular, the combination of (a) being certain that you’re in an infinite world and (b) not knowing how to reason about infinite worlds seem an especially insulting double-whammy (thanks to Paul Christiano for suggesting this juxtaposition). But here, again, I want to make noises similar to those I made about pascal’s mugging-ish examples, namely: yes, but aren’t we also a bit used to “uh oh: this otherwise attractive view, developed in the context of finite cases, says weird/unclear things in infinite cases, and maybe becomes obsessed with them?”. See, e.g., expected utility theory, population ethics, and so on. Indeed, totalism about population ethics becomes similarly obsessed with infinities (that is, in creating/influencing them, rather than believing in them), and it becomes similarly confused about how to compare them. And in both cases — SIA, and totalism — the obsession-confusion combo seems no accident. Infinites are confusing things, especially for views that wanted to rely on a relatively straightforward, everyday usage of “more” or “bigger.” But infinities are also confusingly big. So if you’re excited about big stuff, it’s easy to end up obsessing about something confusing.
Indeed, in general, the fact that these problems with SIA — e.g., Pascal’s mugging-type cases, infinity issues — are so structurally similar to problems with expected utility theory and totalism seems, to me, some comfort. They aren’t good problems. But in my opinion, it’s good (or at least, respectable) company. And more, it’s company that seems to keep being useful, at least as a first pass, despite these problems lurking in the background. Perhaps SIA is the same.
I’ll say one other word in SIA’s defense re: certainty about infinities. If you really get into the vibe of “I am a particular possible person-in-my-epistemic-situation, who didn’t have to exist,” and you actually think of the world as drawing you out of a hat of possible people like that, then it doesn’t seem that crazy to think that the fact that you got drawn effectively seals-the-deal in terms of whether there was an infinite number of draws. The hat of possible people-in-your-epistemic-situation, after all, is presumably extremely infinite. So what, you think you were going to just happen to get drawn after a finite number of draws? Har har. Yes, there are a few people in that situation, in cases of finitely-many draws. SIA does indeed make those people infinitely wrong. But, like, that’s not going to be you. You’re not going to actually get drawn in those worlds. So you don’t need to worry about being wrong in them. To a first and infinitely close approximation, that is, you only exist in infinite-draws worlds. So certainty about them is fine.
What about other objections to SIA? A common one is that you don’t learn anything new in Sleeping Beauty (you knew, on Sunday, that you were going to wake up regardless, and you were a halfer then): so why should you update upon waking? My take here is basically just: once you’re thinking about it in terms of person-moments, and once you get into SIA’s basic ontology, this problem goes away. Sure, some person-moment-in-my-epistemic-situation was going to exist either way: but that doesn’t mean that “I” was going to exist either way. (My suspicion is that something similar is the right thing to say about Roger White’s “Generalized Sleeping Beauty” problem. E.g., if you grok that waking up is actually evidence, the problem resolves. That said, I haven’t really worked through it.)
There are other objections as well. Paul Christiano, for example, pressed the objection in conversation that SIA’s reliance on some notion of “counting” the number of observers isn’t going to play nice with quantum mechanics or brute physical notions of probability; and that it’s going to break in e.g. cases where you split a computer running a given mind into two computationally-identical slices to different degrees (Bostrom discusses cases like this here). My reaction here is: yeah, sounds like there are some issues here, but also, aren’t we probably going to need some way of counting observers in at least some cases – for example, the number of observers in God’s white rooms, or the number of wakings in Sleeping Beauty, or the number of attendees at your weekly bingo session? I do think things will get gnarly here: but without having dug into it much at all, I’m inclined to think it’s the type of gnarly that lots of views (including SSA) will have to deal with.
Does SIA make bad empirical predictions? That would be pretty damning if so. For example, if it turns out that the universe actually is definitely finite in a fundamental sense, there’s a strong temptation to throw SIA out the window, unless we’ve somehow revised it to get rid of its certainty about infinities. But we might wonder about whether we can get started with this whole out-the-window process earlier. For example, does SIA predict that there actually should be a Graham’s number of observers-like-us hiding in my closet? Like, why isn’t our world chock full of observers-like-us in every possible nook and cranny? How come I can move without bumping into a copy of myself?
This feels like the type of objection that might move me a lot if we worked it out, but it also seems a bit tricky. In particular, SIA is only excited about observers-in-our-epistemic-position. And our epistemic position is: huh, doesn’t seem like there are that many observers-like-me hiding in my closet. So it’s not actually clear to me that SIA does predict that I’d see such observers after all. That said, I haven’t thought very much about this, and I wouldn’t be surprised if there’s a good objection in this vicinity.
Does SIA imply dynamic inconsistencies? Yeah, I think it does in some cases. For example, what was up with that fifth-ing thing? That sounds like the type of thing you’d want to self-modify right out (and in particular, it sounded like a kind of “double counting,” at least from the perspective of the prior — thanks to Paul Christiano and Carl Shulman for discussion). But as I mentioned in part 3, I think lots of views involve dynamic inconsistencies, and my generic, first-pass response is just: if you’re worried about dynamic inconsistencies, go updateless. That’s what it’s for. (Or at least, that’s how I tend to think about it. Indeed, in my world, one not-especially-charitable gloss on updateless-type decision theories is “whatever the heck I have to say about decision theory in order to have a consistent policy.”) That said, I could imagine getting more fine-grained about when exactly different sorts of dynamic inconsistencies are more or less damning, so there’s more to be said.
Even if we accept “just go updateless” as a response, though, I can imagine protest: Joe, if you’re ultimately going to just go updateless and start acting like e.g. an altruistic EDT-ish totalist SSA-er, or some such, then this whole thing about how SIA > SSA is a big sham (Carl Shulman suggests the analogy: “I’m a Democrat but I oppose every single policy position of Democrats”). And maybe something like this critique is ultimately going to apply. However, my current view on this is: first, I don’t want to get ahead of myself on policy positions. As I mentioned in the section on betting, I have yet to go through and really tease apart the relationships and interactions between what seem to me an importantly diverse array of variables, here (e.g., EDT vs. CDT, updateless vs. not, altruism vs. not, averages vs. totals, bounded vs. unbounded, and so on). And I don’t think “just moosh the variables together already” is a heuristic that works in service of clarity. I think that others may well have achieved genuine clarity on this front; but I haven’t, yet, and I don’t want to jump the gun.
Beyond this, though, even if I ultimately want to go updateless, EDT-ish, altruistic, totalist, or whatever, some of my readers might not — and such topics seem, naively, like importantly additional discussions, implicating a lot of additional desiderata. And regardless of the outcome of such a discussion, updateful, CDT-ish, selfish, average-ish, etc people can still wonder whether they should believe the doomsday argument, whether they’re highly likely to live in an infinite world, whether telekinesis works, or whatever. That is, naively construed, anthropics presents itself as a question about what we can learn about our situation from a certain type of evidence. And I don’t want to obscure this question by taking on board a bunch of additional ethical and decision-theoretic assumptions, and then saying that granted those assumptions, the question doesn’t matter — especially when other people, who don’t accept those assumptions, are still going to ask it.
I’ll mention one other objection to SIA: namely, that it doesn’t feel especially intuitive as a “conception of yourself and the world.” That is, it would be nice if our theory of anthropics made sense as a picture of what we are and what the process for deciding whether or not we get created looks like. We could then return to this picture when we get confused, and thereby keep a closer grip on why, exactly, we’re reasoning this way in the first place. Indeed, absent such a grip, it can feel like we’re just scrambling to draw the right sort of line through the curve of the cases. We’re making things up as we go along, with no tether to a view about the way the world is.
In part 1, I’ve presented one SIA-ish “story” about this: namely, that you are a “possible person-in-your-epistemic-situation,” who gets pulled from the “hat” of such people, with a likelihood proportional to the number of “pulls” a world implies. But I think it’s pretty reasonable to look at this picture and say: what? It’s better, I think, than SSA’s “God goes hunting for you specifically in the hat, then throws you randomly into the reference class” picture, but it’s not exactly clean and pretty. What exactly are these hats? What is the space of possible people? How does the drawing process work? Does “who I am” change as my epistemic-situation changes? What’s all this really about?
(I can also imagine a different sort of objection to this sort of story — one that attempts to get some sort of likelihood ratio on your existing, conditional on SIA’s metaphysics, vs. SSA’s metaphysics. That is, SIA essentially imagines that you’ve won some ridiculous “possible person who got pulled from the platonic realm” lottery. Whereas SSA imagines that you’re a special snowflake and that God was dead set on creating you — or, alternatively, that you’re the world spirit who was going to experience whoever God created. And which metaphysics posits that a more unlikely event has occurred?)
I don’t have especially good answers re: better SIA stories, but here’s at least one alternative, which might be more intuitive for some. On this story, SIA is centrally about a kind of “principle of indifference” about who you are, applied to all the people you might be (including people in different possible worlds), but weighted by probability that those people exist (thanks to Katja Grace for suggesting formulations in this vein). That is, SIA notices that it exists in its epistemic situation, then says: “Ok, who am I?” It then looks at all the people in that epistemic situation who might exist, and tries to not-be-opinionated-with-no-reason about people like that who are equally likely to actually exist. Thus, in Sleeping Beauty, SIA reasons: “Ok, I’ve woken up. So, which person-moment am I? Well, I might be Heads-Monday, I might be Tails-Monday, and I might be Tails-Tuesday. Heads-Monday is 50% likely to exist, and Tails-Monday and Tails-Tuesday are both 50%, too. So, they’re all equally likely to exist. Thus, with no special reason to favor any of them, I split my credence evenly: 1/3rd on each. Thus, I’m 1/3rd likely to be in a Heads world.”
And if the original coin had been weighted, say, 25% on Heads, and 75% on tails, SIA would adjust accordingly, to make sure that it stays equally likely to be equally-likely-to-exist people-in-its-epistemic-situation: “Ok, Heads-Monday is only 25% likely to exist. Whereas Tails-Monday and Tails-Tuesday are both 75% likely. If they were all equally likely to exist, I’d be 1/3rd on each; but actually, the tails people are 3x more likely to exist than the heads person. So, upweighting each of those people by 3x, I end up at 1/7th on Heads-Monday, and 3/7ths of each of Tails-Monday and Tails-Tuesday. Hence, 1/7th on Heads.”
That said, this sort of framing raises the question of why you don’t update again, once you’ve decided that tails is more likely than heads. That is, granted that tails is 2/3rds, it’s now 2/3rds that Tails-Monday and Tails-Tuesday exist, and only 1/3rd that Heads-Monday does. So why doesn’t SIA reason as follows? “Ah, actually, the tails people are each twice as likely to exist as the heads people. So, instead of 1/3rd on each, I’ll be 2/5ths on each of the tails people, and 1/5th on the heads person. But now, actually, it looks like tails is 4/5ths and heads is 1/5th. So actually, instead of 1/5th on heads person, I’ll be 1/9th…” and so on, until it becomes certain of tails. So while I think this framing has advantages over the “possible people in the platonic hat” framing, it also risks a kind of instability/converge towards false certainty. Partly for that reason, I currently don’t lean heavily on it.
I’ll mention one other possible reframing, which I haven’t worked out in detail, but which feels spiritually similar to me. Consider a case where God flips two coins. The first coin decides how many people he creates: if heads, one; if tails, two. The second coin to decide who he creates, if there’s only one person. if Heads-Heads, Bob; if Heads-Tails, Alice. If Tails-Heads or Tails-Tails, though, God creates both Alice and Bob.
Suppose you wake up and find that you are Bob. Bob exists in three out of these four cases, all of which are equally likely, so it seems very natural, here, to be 1/3rd on each, and hence 1/3rd on the first coin having landed heads. We don’t have to do any special sort of anthropic updating in favor of worlds with multiple Bobs — there aren’t any. Rather, here, it’s SSA that does weird anthropic stuff: in particular, it downweights the Coin-1-Tails worlds, because in those worlds, it’s 50% that you would’ve been Alice.
SIA’s very natural, non-anthropic type of reasoning here feels pretty similar, though, to the type of thing that SIA is trying to do all the time, but in the context of uncertainty about who it is. Thus, suppose that in the case above, you wake up and don’t yet know whether you’re Alice or Bob. However, you know what you would think if you knew you were Alice, and you know what you would think if you knew you were Bob: in both cases, that is, you’d be 1/3rd on Coin-1-Heads. And having no special reason to think that you’re Alice instead of Bob, you’re 50-50 on who you are. Thus, given 50% that you should say 1/3rd, and 50% you should say 1/3rd, you say 1/3rd. Maybe thinking about this sort of case can help shed light on SIA’s basic shtick?
Still, I don’t feel like I’ve nailed it in terms of “this is the way the world actually is, such that SIA makes sense.” And I think a lack of a fully intuitive picture here is a significant barrier to really “believing” in SIA, even if it looks good on more theoretical grounds.
There are, presumably, lots of other objections I’m not discussing. Indeed, I feel some worry that in the process of writing this post, I’ve given too much attention to the problems with SSA that feel like they shout from the pages of Bostrom’s book in particular, and that I haven’t gone hunting hard enough for the strongest possible case against SIA (or for SSA’s strongest possible defense). But, this sequence is long enough already. For now I’ll just say: I expect to keep learning more.
Indeed, I can well imagine a version of this post that focuses less on comparing SSA and SIA, and more centrally on a message in the vicinity of “SIA and SSA are both terrible, dear God help us.” That, indeed, is sometimes the vibe I get from Bostrom (e.g. here), though his book devotes more attention and sympathy to SSA in particular. And regardless of the comparative merits of SSA and SIA, I wouldn’t be surprised if really taking infinite cases in particular seriously forces a kind of start-over-and-build-it-all-back-up-again dynamic, rather than some sort of “patch” to an existing view.
Perhaps, then, faced with such unappetizing options, we should refuse to eat. That is, we should recognize that (in this blog sequence at least), we have yet to find a remotely plausible or satisfying theory of anthropic reasoning, and we should “hold out” until we find an “Anthropic Theory X” — one that gets us everything we want (or at least, much more than SIA and SSA do). (Thanks to Nick Beckstead for suggesting this type of response, and for the name “Anthropic Theory X” — a reference to Derek Parfit’s name for the elusive, fully-satisfying theory of population ethics he was searching for).
And to be clear, such a Theory X may already exist. As I’ve tried to emphasize, I’ve only been discussing two basic and prominent views: I haven’t tried to survey the literature as a whole.
And even if no one has invented such a theory yet, it may still be out there, in theory space, waiting for us to find it. In particular, we do not to my knowledge yet have “impossibility proofs” of the type we have in population ethics, to the effect that there is no anthropic theory that will satisfy all of Y constraints we hoped to satisfy (thanks to Nick Beckstead, again, for suggesting this consideration).
That said, I’m not sure we’re so far away from proofs in this vein. In particular, even if SIA and SSA sound like two very specific theories, to which there are presumably many viable alternatives, their verdicts about particular cases seem to exhaust many of the most plausible options about those cases. But yet, it is precisely these verdicts, applied in (at least apparently) structurally identical contexts, that lead to some of their worst results.
Consider, for example, Sleeping Beauty. What, actually, are you going to say in Sleeping Beauty, if not ½, or ⅓? (Let’s leave aside “fifthing” for now, along with “incorporate your uncertainty about anthropics” type moves.) Suppose that you’re like me, and you want say a third, perhaps because you’re moved by basic, compelling, and fairly-theory-neutral arguments like “you should be 1/4th if you wake up on both days no matter what, and then if you learn that you’re not Heads-Tuesday you should clearly end up a thirder.” Perhaps, indeed, you’d be inclined to put some of the premises of those arguments into your “impossibility proofs” as one of the Y constraints. (Are “impossibility proofs” and valid arguments especially different? The former has an aura of technical finality and “having really created knowledge.” But we’ve had impossibility proofs for ages that there is no Theory of X of Socratic Immortality, such that (1) Socrates is a man, (2) All men are mortal, and (3) Socrates is not mortal.)
Ok, so say you craft your candidate Theory X to say a third. But now make it a zillion wakings if tails instead. It’s the same case! There’s no magic about the number “a zillion” — or at least, “no magic about the number a zillion” seems like it could also be one of those Y constraints that we could put into our impossibility proofs. But now Theory X is getting pretty darn confident about tails. Presumptuously confident, you might say.
Indeed, as I tried to emphasize above, the Presumptuous Philosopher and its variants are just science-ified versions of a zillion-wakings Sleeping Beauty. We can futz about whether it matters that e.g. the prior is some kind of objective frequency in Sleeping Beauty vs. some build up of empirical evidence about fundamental reality in a more scientific case. And maybe there’s stuff to say here, and other differences to bring out. But ultimately, the basic thing that thirding does, regardless of its justification, is update towards worlds where there are more people in your epistemic situation. And this is also the basic thing that many of the most prominent objections to SIA get so worried about.
Thus, to go further: make it an infinite number of wakings, if tails. Uh oh: sure seems like this should be an update towards tails, relative to a merely zillion-waking world. After all, we can count at least a zillion wakings in the infinite world pretty easily (just label them starting from 1). Indeed, sounds like we might be in for it re: for any finite number of wakings, being more confident than that. But that sounds like certainty.
Obviously, there’s more to say here, especially about infinite cases. The thing I want to point at, though, is that the distance between “thing we really want to say” and “thing we really don’t want to say” isn’t necessarily very “theory laden.” Rather, it looks a lot like we like/want a given type of result in one case, and then we hate/don’t want that same type of result, in some other case with a different vibe, or a greater feeling of “extremity,” or a more serious sense of real-world implication. This dynamic currently inclines me towards pessimism about finding an especially satisfying theory X, at least of a standard kind, that avoids both SIA and SSA’s problematic implications. I think SIA and SSA, and their closely-similar variants, might be covering more of the conventionally-plausible ground than it initially appears. Perhaps, indeed, those who seek theory X do better to try to reconceptualize the whole terrain, to “dissolve the problem entirely,” rather than to approach it on its own terms (Armstrong’s Anthropic Decision Theory has a bit of this flavor). But even then, for whatever reconceptualized equivalent of the question, one wonders: ½, or ⅓?
I’ll close with a brief discussion of implications. What would SIA, if true, say about our real-world situation?
People have said various things about this. A classic thing is that we stop believing in the Doomsday Argument, which sounds pretty good. Indeed, one update I’ve made in the course of writing this post is towards not-buying SSA-ish arguments for doom soon.
But actually, maybe SIA suggests it’s own Doomsday Argument: namely, that probably the reason we don’t see much intelligent life out there is because ~all life kill itself after reaching our stage, rather than ~never reaching our stage at all, because this story makes it more likely that there’s lots of life out there at our stage and hence like us (see Grace here; and some related discussion here). But actually, maybe SIA doesn’t say this, and instead says we should update towards overwhelming probability of being in a simulation run by a maximally powerful civilization devoting ~all of its resources to simulations of us-in-particular (thanks to Carl Shulman for discussion; see also section 4 here for more on SIA and simulations). But actually, maybe SIA doesn’t say any of this, because it has a seizure immediately after becoming certain that we’re in an infinite universe (and presumably, the biggest possible infinite universe? Something something modal realism Tegmark-Level-4 Ultimate Multiverse? Or something bigger, with impossible worlds too?).
Overall, I feel pretty far away from any kind of clear picture of what SIA would actually imply, especially given the seizure thing. And indeed, as with the acausal wackiness I discussed in my last post, it feels to me like we’re at a sufficiently early stage in reasoning about this stuff that we ought to tread very carefully, and avoid making updates that seem pretty conventionally silly or extreme. What’s more, I’ve been explicitly setting aside stuff about decision theory, copy-altruism, and so on, all of which could well change the practical game entirely in terms of “what does this imply,” and maybe restore various types of “normality.” For example, as I gestured at above above, if you’re updateless in a way that counteracts some of SIA’s implications (for example, if you’re a copy-altruistic EDT-er who self-modifies to avoid fifth-ing), you may end up acting like an (EDT-ish) SSA-er in lots of cases, even if you like SIA in principle (thanks to Carl Shulman, Paul Christiano, and Katja Grace for discussion).
That said, as with the acausal wackiness, I don’t think “cool let’s just ignore this stuff entirely” is the right response, either. In particular, anthropics — at least, naively construed — purports to identify and make use of a form of evidence about the world: namely, for SIA, the evidence we get from the fact that we exist; and for SSA, the evidence that we get from the fact that we exist as these people in particular, as opposed to others in the reference class. This form of evidence is often overlooked, but on both of these views, it can end up an extremely powerful clue as to what’s going on (hence, presumptuousness — and views that aren’t presumptuous in this way struggle to make basic updates/conditionalizations in cases like God’s coin toss with equal numbers). Neglecting anthropics as a category of consideration therefore risks missing out on centrally important information — including information it might be hard to get otherwise (for example, information about great filters, simulations, multiverses, and so on). And even if we don’t see any immediate uses for this information (e.g., “Ok but tell me right now what practical difference this is going to make”), it seems useful to have on hand. A more accurate basic picture of your existential situation as a whole, for example, seems pretty robustly worth having. And some of us are also just curious.
What’s more, doing anthropics badly has costs. You end up confused about the doomsday argument, for example. You end up reasoning badly about the fine-tuning of the universe. You end up wondering whether your metaphysical essence is compatible with being a chimp, or a bacterium. At the very least, we need some sort of anthropic hygiene, to avoid making these sorts of errors. And the line between “avoid basic errors” and “make actually-important updates” isn’t especially clear.
Overall, then: I currently think SIA is better than SSA. SIA still has problems, though, and I’m not especially sure what it implies in the real world. We should try to figure out a better theory (the need to handle infinite cases seems especially pressing), and perhaps there is one out there already (as I’ve said, I haven’t looked hard at the alternatives). In the meantime, we should tread carefully, but stay interested in understanding the implications of the theories we have.