tl;dr: many effective altruists and rationalists seem to have key misunderstandings of anthropic reasoning; but anthropic probability is actually easier than it seems.
True or false:
- The fact we survived the cold war is evidence that the cold war was less dangerous.
I'd recommend trying to answer that question in your head before reading more.
Have you got an answer?
Or at least a guess?
Or a vague feeling?
Anyway, time's up. That statement is true - obviously, surviving is evidence of safety. What are the other options? Surviving is evidence of danger? Obviously not. Evidence of nothing at all? It seems unlikely that our survival has exactly no implications about the danger.
Formalised probabilities beat words
The problem, in my view, is that we focus on true sentence like:
- If we're having this conversation, it means humanity survived, no matter how safe or dangerous the cold war was.
And this statement is indeed true. If we formalise it, it becomes: P(survival | conversation) = 1, and P(survival | conversation, cold war safe) = P(survival | conversation, cold war dangerous) = 1.
Thus our conversation screens off the danger of the cold war. And, intuitively, from the above formulation, the danger or safety of the cold war is irrelevant, so it feels like the we can't say anything about it.
I think it's similar linguistic or informal formulations that have led people astray. But for the question at the beginning of the post, we aren't asking about the probability of survival (conditional on other factors), but the probability of the cold war being safe (conditional on survival). And that's something very different:
- P(cold war safe | survival) = P(cold war safe)*P(survival | cold war safe)/P(survival).
Now, P(survival | cold war safe) is greater that P(survival) by definition - that's what "safe" means - hence P(cold war safe | survival) is greater than P(cold war safe). Thus survival is positive evidence for the cold war being safe.
Note that this doesn't mean that the cold war was actually safe - it just means that the likelihood of it being safe is increased when we notice we survived.
Doing anthropic probability, properly and easily
The rest of this post is obsolete; see the post here for the most recent version of anthropic probability in Fermi situations. Decision theory considerations cannot be avoided.
I've recently concluded that anthropic probability is actually much easier than I thought (though the work in this post is my interpretation rather than a generally accepted fact, currently). Don't worry about reference classes, SIA, SSA, and other complications that often seem to come up in anthropic reasoning.
Instead, start with a prior chosen somehow, then update it according to the evidence that a being such as you exists in the universe.
Let's look how this plays out in a few test cases.
Fermi paradox and large universes
First, we can apply this to the probability that life will appear. Let's assume that there is a probability x for life appearing on any well-situated terrestrial-like planet around a long-lived star.
Then we choose a prior over this probability x of life appearing. We can then update this prior given our own existence.
Assume first that the universe is very small - maybe a few trillion times bigger than the current observable universe. What that means is that there is likely to be only a single place in the universe that looks like the solar system: humanity, as we know it, could only have existed in one place. In that case, the fact that we do exists increases the probability of life appearing in the solar system (and hence on suitable terrestrial-like planets).
The update is easy to calculate: we update our prior P by weighting P(x) with x (which is the probability of life on Earth in this model) and renormalising (so P(x | we exist) = ). This updates P quite heavily towards large x (ie towards a larger probability of life existing).
Now suppose that the universe is very large indeed - 3^^^^3 times larger than the observable universe, say. In that case, there will be a huge number of possible places that look identical to our solar system. And, for all but the tiniest of x's, one of these places will have, with probability almost 1, beings identical with us. In that case, the update is very mild: P(x | we exist) = , where w(x) is the probability that a humanity like us exists somewhere in the universe, given that the probability of life existing on a given planet is x. As I said before, for all but the tiniest values of x, w(x) ≈ 1, so the update is generally mild.
If the universe is actually infinite in size, then P(x) does not get update at all, since the tiniest x>0 still guarantees our existence in an infinite universe.
There is a converse - if we take a joint prior over the size of the universe and the probability x of life, then our existing will push towards a large universe, especially for low x. Essentially, us existing is evidence against "small universe, small x", and pushes the probability away from that and into the other possibilities.
The simulation argument
The simulation argument is interesting to consider in this anthropic formulation. In a small universe, we can conclude that we are almost certainly in a simulation. The argument is that the probability of life emerging exactly as us is very low; but the probability is higher of some life emerging, and running lots of simulations which would include us. The key assumption is that any advanced civilization is likely to create more than one simulated alien civilizations.
Hence this updates towards simulation in small universes.
If the universe is large, however, we are more likely to exist without needing to be in a simulation, so the update is far smaller.
Dealing with the Doomsday argument is even simpler: there ain't such a critter.
This anthropic theory cares about the probability of us existing right now; whether there is a doom tomorrow, or in a trillion years, or never, does not change this probability. So there is no anthropic update towards short-lived humanity.
But what about those arguments like "supposing that all humans are born in a random order, chances are that any one human is born roughly in the middle"?
Normally to counter this argument, we bring up the "reference class issue", asking why we're talking about humans, when we could instead be talking about all living beings, all sentient beings, all those who share your family name, all left-handed white males in the 35-45 year range currently writing articles about anthropics...
Let's be more specific: if I put myself in the reference class of males, then I expect that males are more likely than females to die out in the future. Any females friend of mine would conclude the opposite.
This is enough to show that a lot of fishy things are going on with reference classes. I my view, the problem is more fundamental: imagine that (hypothetical) moment when you're thinking "I know I am a human, but I will now open my eyes to know which human I am." In that very moment, I am... identical to any other hypothetical human who hasn't opened their eyes either. In other words, I am now part of a class of duplicates.
Thus, as I will argue in a later post, reference classes are fundamentally objects of decision theory, not of probability theory.
Let's apply this to the famous Sleeping Beauty problem. A coin is flipped and, if it comes up heads, you are to be awoken once more; if if comes up tails, you are to be awoken twice more, with an amnesia potion in between the two awakenings.
By the argument I presented above, the prior is obviously 50-50 on heads; in both the heads and tails universes, we are going to be awoken, so there is no update: after awakening, the probability of heads and tails remains 50-50.
Note a few key differences with standard SSA, though. First of all, there are no references classes here, which were the strongest arguments against the original SSA. Secondly, there are multiple identical agents, so the 50-50 odds may be "true" from some abstract point of view, but as soon as you have to act, this becomes a question of decision theory, and decision theory will most often imply that you should behave "as if" the odds were 2/3-1/3 (the "thirder"/Self-Indication Assumption answer).
So the odds are 50-50, but this doesn't mean that Sleeping Beauty has to behave in the way that the halfers would naïvely expect.
What of the possibility that we are a Boltzmann brain (a human brain-moment created by random thermal or quantum fluctuations)? If the universe is large or infinite, then it is almost certain to contain us as a Boltzmann brain.
And this would seem to stop us from reasoning any further. Any large universe in which there were some fluctuations would get the same update, so it seems that we cannot distinguish anything about the universe at all; the fact that we (momentarily) exist doesn't even mean that the universe must be hospitable to human life!
This is another opportunity to bring in decision theory. Let's start by assuming that not only we exist, but that our memories are (sorta) accurate, that the Earth exists pretty much as we know it, and that the laws of physics that we know and, presumably, love, will apply to the whole observable universe.
Given that assumption, we can reason about our universe, locally, as if it's exactly as we believe it to be.
Ok, that's well and good, but why make that assumption? Simply because if we don't make that assumption, we can't do anything. If there's a probability that you may spontaneously vanish in the next microsecond, without being able to do anything about it - then you should ignore that probability. Why? Because if that happens, none of your actions have any consequences. Only if you assume your own post-microsecond survival do your actions have any effects, so, when making a decision, that is exactly what you should assume.
Similarly, if there's a chance that induction itself breaks down tomorrow, you should ignore that possibility, since you have no idea what to do if that happens.
Thus Boltzmann brains can mess things up from a probability theory standpoint, but we should ignore them from a decision theory standpoint.