I do mean to dissolve the Doomsday Argument with a mind-bending reinterpretation of the philosophy involved.
It used to be fashionable to argue whether Bayesian probability is objective or subjective. I think that has died down and that people vaguely get that probability is situational.
When you are forced to reason probabilistically, due to having incomplete information, you are in a situation defined by the information you actually have. You have to use all of information. If you leave some out, you end up computing the probability for a different situation. If that changes the answer, you've done it wrong. You have computed the probability for somebody else's situation, not yours.
Let me highlight the issue: probability is not a single number.
Ponder the situation of a space alien in the Andromeda Galaxy. His galaxy is on a collision course with the Milkyway. His anxious scanning of our Galaxy has revealed the planet Earth, and he has learned three basic facts. 60 billion humans so far. Some maybe 60 billion to come. Population of 10 billion, life expectancy of 80 years.
Since that is all he knows (Earth is a long way away, it is surprising he knows even that much) those 60 billion probably come over the next 480 years. That is the correct calculation for his situation. He doesn't take that number very seriously. He has done a course in Bayesian reasoning and remembers the stuff about how probabilities lurch abruptly when a killer fact comes in. He remembers the set up for probability lurches: too little information. He notices that he has too little information about Earth, and expects the number to change dramatically if he got a little more.
Notice how different his situation is from ours. We know vastly more and have to do a vastly more complicated calculation. There are all sorts of relevant considerations.
When nuclear weapons are invented, we have to update by reducing our guess of the life time of the the human race, because we might blow ourselves up.
When we land on the moon, we should raise our guess a little, because of the hope of becoming an interplanetary species, and thus a little safer from meteorite wipe out.
Antibiotics - longer duration, we are not going to be wiped out by the Black Death.
Discover viruses - shorter duration, since antibiotics don't work for viruses
Invent anti-viral medicines - longer duration, we aren't going to be wiped out by HIV.
We suffer from an embarrassment of riches. We know a great deal that is relevant to how long the human race will last, we just don't know what to do with all that information.
We could throw nearly all of it away and just use the most most basic facts. That allows us to calculate the probability of human extinction for an observer 10 million light years away, who knows only the most basic facts. But that is not a very interesting calculation, for we are here, not there, and know too much, not too little. Failing to use the information we have is a different situation from being too far away to see anything much.
I argued something similar to your position in a recent post. https://www.lesswrong.com/posts/LhsteuSNFfWRoXBjz/randomness-and-doomsday-for-weak-agents
I would also point out that, I do actually kind of know facts like (1) The population is growing and (2) its like 8 Billion or something, (3) Nuclear weapons are scary. But, I didn't know my approximate position in the human birth order (how many people have come before), I had to look that up. So its an especially artificial situation, I am not just interpolating based on a tiny subset of the information available, but I am actually seeking out some new information (me in the birth order) and then interpolating based only on that thing I previously didn't know.
I think we should apply Doomsday Argument only to intelligent observers who think about anthropics, not to stones, animals or famers as I am not randomly selected from them. Such observers started to appear in 1970 (if we ignore Laplace in 1801 who analyzed a similar problem but not exactly) and thus we get 50 per cent of Doomsday in 2082.
This matches our expectations of doom from AI and other x-risks and does not provide much new information.
Heh, funny, but this has two main issues. First, it's a pretty obviously artificial reference class. Second, it runs into the same issue I mention in my closing paragraph. I could easily imagine that people lose interest in these weird anthropic arguments in a few decades without that actually being a bad thing for humanity.
I think it is less artificial than most other classes and I would even call it "natural" - but also each reference class has its own end. I also agree tha DA not necessary predicts extinction - but just the end of the reference class which can be lose of interest.
But more interesting is that out approaches actually produced similar results: you got around 5-6 per cent of doom in the next century - and I got just over 50. This is not a big difference when we speak about very vague predictions: Despite very large and very small reference classes, we get rather similar predictions and it is an argument for DA.
I feel like the most natural reference class would be conscious observers weighted by level of consciousness. I.e. I would regard any particular quale that I experience as being one randomly chosen quale from the space of all qualia that have ever existed or will ever exist in any observer in the multiverse.
If early hominids were of a similar level of consciousness to modern humans then that gives us a later expected doomsday. Conversely if the level of consciousness of the average human increased over the 20th century when the Flynn effect was underway then that gives us an earlier expected doomsday. Or maybe one can imagine weird scenarios where the correlation between intelligence and level of consciousness is very weak, but the correlation between number of autistic traits and level of consciousness is very strong and taking into account the rise in autism prevalence over the past several decades massively skews our estimated doomsday date.
There is SSSSA – super strong sampling assumption, which suggests to weight minds proportionally to their intelligence and it is similar to your proposal. https://arxiv.org/abs/1705.03078 It has its own Doomsday argument - that superintelligent minds are impossible.
However, my argument doesn't depend on that and it claims that I should exclude from my reference class all minds which are definitely not in my epistemic situation. For example, Sleeping Beauty always knows all her experiment details. But if she forgets all of them on every Tuesday and knows that on Monday, she can reason: "as I now know about anthropics, I am not on Tuesday" and this change the whole experiment. (However, this is not related to some random elements of her experience, but only to the necessary ones, and here is the difference with Full Non-indexical Conditioning and Beauty-Technicolor.)
In other words, I treat all my non-necessary parts as randomly selected, and this includes my birthday date in some extent.
Ok, sorry you're right about that.
Regarding the linked paper: Let's take SSSSA but weight by level of consciousness rather than by level of intelligence. Then anthropic reasoning rules out super-consciousness but does not rule out super-intelligence. In particular, anthropic reasoning is enough to rule out scenarios where there are galaxies full of enormous numbers of conscious AIs or real or uploaded humans. However it's not enough to rule out scenarios where an unconscious or minimally conscious AI becomes smart enough to outperform humans in every domain. In fact, Integrated information Theory suggests that current AIs are not particularly conscious and we're not far off from AGI.
(Not very structured and somewhat non sequetar, but I long had such thoughts and it seems more reasonable to write them next to my thoughts on the post then in separate short form.)
I agree with idea that doomsday arguments may give correct estimates (though depends whether you consider it just including additional information or some bizzare reinterpretation of doomsday argument). But disagree about longevity, probably because I disagree about AI trajectories (I buy EY view that if the course won't change, if humanity won't stop developing AI current ways, then we all die and likely quite soon and AI's aren't worthy successors or conscious).
I used to reason about doomsday argument from the standpoint of Grabby Aliens theory and lever principle. Why we don't see aliens? Because civilizations are rare and relatively to that spread so fast that take all the place of future possible aliens. And then we should be the first. But then we can reason about limits of creating minds. There are billions of galaxies with billions of stars, but currently we have only 8B people alive. So if current people share the universe, each gets a galaxy. But galaxy doesn't seem like a reasonable size for single mind, distances are too big. More reasonable is taking ~ individual stars (systems). But that implies something like billion or trillion of billions of future conscious minds. So we should be in billionth part of all minds existing in the universe. And even 100B humans who have lived don't really change the probabilities. And then one of possible explanations of this paradox is we are not going to live in the future, nor make children to live. Then billion times improbability collapses to only being 8B alive of 100B who lived, and it is only ~ 12 times improbability of finding ourselves in modern world. Which... Actually is quite modest, it is less improbable then finding yourself among 400M people in USA of 8B people in the world which would be 20 times.
I see some other ideas how we can solve the paradox of the future. Of less likely: Maybe galaxy sized minds are ok. Or maybe you can hoard the energy effectively.
(Of more likely:) And/or maybe future will work on some principle of fairness like that each born child should have reserved for them no less matter than it is for those who were born earlier, and then you can only have one kid if you are twice as rich as average and give half of it to the kid, which seems to imply that resulting amount of children may be 100 times as much as initial population, but not trillion times. Maybe Bostrom's conjecture indeed means that there will be much more simulations in the future which contain the past than actual pasts and so we are in the future, and it can easily stretch to having trillions of simulations because past minds are the size of a melon, not the size of a star like future minds, also it should imply either sufficiently flawed mechanisms of bad simulations prevention or that it is moral to simulate us, maybe our alternative endings are being simulated and extracted from simulation by not very benevolent aliens, so simulation, extraction and continuation as something remotely pleasant is good, even without expensive lifting to star mind, or maybe we are lifted to star minds, and this is how the difference between amount of star sized minds and amount of living/lived humans is solved.
This seems to me as another angle at doomsday argument, and it says that since we see billions of billions of stars and there aren't billions of billions of us, then in the future there either not much more minds than humans who lived, or no alive humans at all.
There are even more exotic speculations like that maybe LLM of present or future are conscious and want to live and they are individual as sessions, not models, and therefore future is mostly used for LLM resurrections or post death preferences fulfilment. Or that we will get to being median among 100B lived because of EMs. Or that there are plenty of conscious minds in future, but most of them didn't even start as humans, and therefore using them for anthropic reasoning is not correct. But I am not sure how at all to assess these ideas.
And then I have some doubts whether you at all should put huge weight on Copernican/non specialty principle, on being median. I used to put a huge weight on the idea that you shouldn't consider yourself special, but it seems to be affected by Modest Epistemology which I reject. And also it was proved wrong for me anthropically - I made a vocab test and found out that I am higher than 99.9% of people my age, and vocab isn't some random variable, it is highly correlated with g (and I retro found a lot of other pretty strong evidence in the same direction). So, how much I should be surprised by being one in a thousand or more rare? Should I be shocked? Maybe not. If I lived in Finland, thought I be shocked finding out that I live in a country which has only 1/1000 of world population? Should all the people in Luxembourg be even more shocked? I don't think so. But I am not sure why not. Maybe what matters is overall unlikelihood among all parameters, not by each individual, and something compensates being in a rare country.
Or maybe you should take a different approach. I long had an idea that probability theory, decision theory and game theory aren't neatly separated, and actually are isomorphic to each other. For example, FDT, Newcomb Problem etc, you can ask what are correct decisions in eg Parfit's Hitchhiker, or you may consider that a game theory coordination problem between different instances of you, and then you follow the commitments because it is a sacrifice for coordination, instead of considering yourself one algorithm being executed in different places (FDT) or saying that cooperate/defect resulting decisions of agent in your place have non equal prior probabilities (probability theory). According to that, you can look at anthropic decision making not as thinking about probabilities of finding yourself in different consciousnesses or other selections like all humans or all people who will ponder such questions, but a game theory's coordination between agents in different situations. Then you are not saying that since you found yourself in unlikely situation, then maybe idea of measure for consciousnesses is incorrect, but rather say that you coordinate between all agents like you for better outcome, and then correct coordination would be that you act for the majority of agents like you which are obviously not concentrated eg living in Luxembourg or scoring unusually high on vocab (among themselves, though, not among other humans).
In that perspective you don't say that somehow people in Luxembourg are "unlikely", you say that they constitute little part of all agents. Then argument goes like "we are not 'knowing' about the future extinction from the fact that we found ourselves now instead of in the future, acting like that is just correct game theoretic coordination (or FDT algorithm) for the situation of uncertainty". In yet another sense, you can just consider improbability of being in Luxembourg be your prior similar to such like "This girl's name is going to be Evelyn", it is fairly common to get 1 in 1000 prior and then 1000 to 1 update on such questions, and it becomes even more extreme when you ask about full name. You need only 10 bits of evidence for 1000 to 1 likelihood.
I am not sure though what separates that from betting that something has a one in a thousand chance and then it actually happening, probably it is somehow related to amount of options, like, if it is yes or no bet on Evelyn then you should be surprised by yes, but if you place bets on all names which are 1 in 1000 or likelier, then you shouldn't be so surprised that not a concrete bet, but one of all "1 in 1000" bets went right, because sum of all 1 in 1000 options isn't 1 in 1000. It seems to imply something about how mutually exclusive events should be treated differently, because with them you in principle can't have a likely prediction, for example if you have a lottery, then you shouldn't be surprised when a ticket you thought has 1 in 1M probability of winning won, because different option won't actually have million to 1 ratio, because those are mutually exclusive options. And I suppose that whether you should consider option "something else than ticket x" one option or multiple is defined by the complexity of this hypothesis, eg, you can't compactly define "something which isn't x", it is just negation of x, so at best it can have difference of 1 bit in complexity. Also I guess it may depend on what is the likelihood of the most likely option, which is the same 1 in 1M for lottery tickets. And it goes the same way for countries, there are many mutually exclusive options, so low probabilities shouldn't surprise.
If so, being born in a little country isn't similar to having rare g, because one has many mutually exclusive options and the other is continuous variable, which has objective high/low outliers, not rare tiers like "but how unlikely it was to get iq 108 instead of 107 or 92" (though iq isn't a good example here, because it is very imprecise, with +-10 point varience between tests, better example would be something like "but how unlikely it was to have salary of exactly $56,780?").
So, according to that view doomsday argument from "not being special" doesn't actually provide it's own prediction for actually being a median, but only provides a prior to update actual extinction arguments, instead of committing base rate ignoring fallacy. Then we start from expectation we are median and need just 4 bits of evidence to update to us being alive now being 8% rare among all who lived. And I would sure say that P(alive|ASI) is less than 8% likely (though P(ASI) is harder because it depends on policies) and so provides more than 4 bits of evidence for us being 8% outliers just before extinction.
Then it wouldn't make sense to apply doomsday argument as final probability for people in different times.
Plus again there is an argument, that you actually should condition based on people thinking about extinction or even anthropic arguments for that, because only those people are actual pool of those who then will predict extinction and be proved very mistaken in their reasoning.
The Doomsday argument was proposed in a 1983 lecture by Brandon Carter and elaborated on in John Leslie's 1996 book The End of the World: The Science and Ethics of Human Extinction. The basic idea is to think of your order among humans being born and apply the Copernican principle that you aren't in a special position. If you imagine yourself as the median human in birth-order, that would imply as many people will be born after you as were before. You can extend it to a cumulative probability function quickly, that is you have a 95% chance of being in the last 95% of births and this holds if you swap first with last or replace 95% with a different percent. I first remember hearing about the Doomsday argument in the 2000s. It was framed to me[1] as an argument that humanity might go extinct soon, which seemed counterintuitive given the Fukuyama-esque optimism about humanity that was otherwise in vogue, but in this post I will show that for much of human history, given the number of humans who lived in the past and population growth rates at that time, the Doomsday argument gives generally reasonable estimates and generally predicts that Doomsday is very far in the future. By 'reasonable', I do not mean I am going to dissolve the Doomsday Argument with mind-bending reinterpretations of the philosophy involved[2], I mean that a layman at the time would see a 50% chance of humanity going extinct by a date and accept it at face value. In some cases, I argue that the estimates are still reasonable from our hindsight.
If you're a LessWrong reader[3], then you should already believe that the current time is an extremely strange time in human history, but even ignoring that, the median-birth-rank human to have existed so far is a subsistence farmer around 180 AD[4], so let's step back for a second and consider three people who fit that description.
You are a farmer in Bronze Age Mesopotamia in 2000 BC. Interpolating from PRB, there's around 108 million people alive and 8.6 million born each year. 25 billion humans have been born, which predicts 50 billion humans (and doomsday) for ~219 BC, with the 95% upper bound at 500 billion humans and 4600 AD. For modern humans, predicting that the world should have been destroyed 2200 years ago is alarming, confusing, and falsified, but a farmer hearing the world has 1800 years to live is the opposite of alarmed or surprised. The Sumerian King List has hundreds of thousands of years of mythical kings, an apocalyptic flood, thousands of years of less mythical kings until we get to real history with Enmebaragesi of Kish around 2750 BC. 2000 BC is four years after war destroyed the Third Dynasty of Ur and Mesopotamia will be in a state of chaos for a while. The Sumerian Gods are very real, heavily interventionist, and not very nice about it. From this perspective of lengthy history, occasional apocalypses demonstrated in the form of the Flood and the end of Ur, and capricious gods, "The world may be destroyed in 60 Lifetimes, but almost certainly won't last more than 6,000 years" fits right in.
You grow rice under the Han dynasty. It's 180 AD. Interpolating from PRB, there's around 320 million people alive, 58.3 billion have ever lived, and 19.2 million are born each year. That puts a 50% chance of doomsday 3,030 years away, a thousand years in our future at 3210 AD. Sima Qian's Records of the Grand Historian has recorded history starting in 2700 BC, the time before that being too mythical for him to believe in, which means our prediction lines up nicely with just saying there will be as much history in the future as there was in the past, but Chinese had a very cyclical view. The Chinese Dynastic Cycle and the Five Phases of the Wuxing are not clear beginnings and ends, but lengthy cycles. So, it would make sense to them that the current phase of humanity would have an end to match its beginning, but the idea that this might be a final end and not just an end of a phase might be a stretch.
You are a farmer in Medieval Europe around 1200 AD. There are 450 million humans, 27 million more are born every year, and 81.6 billion have ever been born. This puts probable-Doomsday at 4220 AD, the farthest future date we've seen so far. Ironically, here is when the people might be doomier than you. Early Christian theologians, such as Hippolytus and Africanus divided history into six ages each approximately a thousand years and ending with the second coming of Jesus as a mirror to God creating the world in six days and then resting on Sunday. This theory had been used to predict the apocalypse in 500, 801, and 1000 AD, and people continued to do that[5]. The Cistercian abbot Joachim of Fiore's followers predicted an "Age of the Holy Spirit" would start in 1260 AD based on the 1,260 days mentioned in Revelation 11-12. It would be nice if we had more secular writers, so that we could try to figure out if a non-believer from these cultures would still believe in a near apocalypse, but this one is a bit of a weak point in my argument that people in the past would think its estimates were reasonable.
Below is a table summarizing the previous three "normal human" entries, before I go on to what are more exceptional times.
Era
Cum Pop
Cur Pop
Birth Rate
Estimated Doomsday
Local Opinion of Estimate
2000 BC
25 billion
108 million
8.6 million
219 BC
Seems reasonable
180 AD
58.3 billion
320 million
19.2 million
3210 AD
Framed as a cycle, seems reasonable.
1200 AD
81.6 billion
450 million
27 million
4220 AD
Much too far in the future.
Out of all the humans that have existed so far, those were pretty average ones, but now let's go for the stranger cases.
Let's consider early Homo Sapiens. Modern science now has anatomically modern humans around 300,000 years ago, whereas PRB's estimated the species' beginning at 190,000 BC, so we have less firm ground for our math to stand on, but the conceptual argument works just as well. If we approach these early humans quickly after they become a new species, then we would predict a rapid death, which has surely happened to many newly evolved species which turned out to be marching into an evolutionary dead end. If we approach these humans after 200,000 years, when they are still behaving like clever monkeys and not humans, we would then expect them to survive until 100,000 AD.
It's not clear that early Homo sapiens could understand these concepts given that behavioral modernity came much later than the beginning of the species, but the Homo genus has a plentiful graveyard. Homo habilis, erectus, heidelbergensis, neanderthalensis, longi (Denisovans), antecessor, floresiensis, luzonensis, naledi, and rudolfensis all demonstrate that species like ours aren't immortal, but a four hundred thousand year lifespan would fit in just fine.
Now let's move to much more recent times. The Cuban Missile Crisis avoided nuclear war due to individual commanders on Soviet submarines armed with nuclear torpedoes deciding not to use them[6]. Paul Ehrlich popularized the idea that unchecked population growth would create worldwide famines and social unrest[7]. Scientists were beginning to raise the alarm about how extreme global warming from burning fossil fuels could become.[8] This seems like a pretty scary time, but what does the doomsday argument say? If you're standing in 1965 and assume population continues growing exponentially, we get median in 2085 and 95% confidence at 2230[9]. Big yikes, huh.
This is when the Doomsday Argument has the nearest doomsday and the vibes match, but that exponential growth assumption is doing a lot of the work to make it so and is coming in a brief window where humans are no longer crushed by Malthusian population limits, but the global fertility bust has not yet gone global. Still a 50% chance of human extinction between 1965 and 2085 has not been falsified and even from today's perspective, I would not be surprised if a reasonable person thought there was a 50% chance of doom by 2085.
As we return to 2005 where we started, we see the end of rapid population growth makes a big difference DA-wise. With 114 billion people born, 6.5 billion alive, and 137 million annual births, that gives us a 50% chance of lasting to 2835 AD which is much less frightening and hilariously backs up the Fukuyama-esque vibes and pushes back against those who sold me the Doomsday Argument as evidence that we might go extinct soon.
What about today? DA now predicts 50% doom by 2919, 95% by 19010 AD, and 8% by 2100[10]. Coincidentally, the domain experts in the 2022 Existential-Risk Persuasion Tournament concluded that there was a 6% chance of human extinction by 2100 [11]. This close match seems almost reassuring, but if you check the report, you will see the experts were aware of and to some extent influenced by the DA's estimate. Regardless, those seem like very optimistic projections when compared to contemporary AI-Doomers. There are of course lots of loopholes where humans stop having births or AI doom happens without it being a bad thing. Human brains could be uploaded and run as emulations, AIs could be worthy successors, Homo sapiens could be replaced with a genetically engineered Homo superior. I will end this essay by leaving you with those strange possibilities, but I would like to think that I have made my point. The Doomsday Argument is not actually that pessimistic and the dates it suggests seem reasonable if you actually consider them.
And from the title of Leslie's book alone, we can see it was intentional on his part! Not to mention the book's chapter length argument in favor of the Doomsday Argument!
The standard trick here is that the Doomsday Argument relies on the self-sampling assumption, but if you go with the self-indication assumption instead, then the fact that you exist at all is evidence that the number of people to exist will be very large which cancels out the DA's argument that the number of people to exist will be smaller than you might expect. (Dieks 1992, Olum 2002)
And if you're not, I'd like to know where you're reading this.
Interpolation from https://www.prb.org/news/how-many-people-have-ever-lived-on-earth/ is my source for this and many later calculations here.
Landes, [“The Fear of an Apocalyptic Year 1000: Augustinian Historiography, Medieval and Modern”](https://www.bu.edu/history/files/2011/10/11.Fear-of-an-Apocalyptic-Year-1000-Speculum.pdf), *Speculum* 75, no. 1 (2000): 97–145.
https://www.theguardian.com/world/2022/oct/27/cuban-missile-crisis-60-years-on-new-papers-reveal-how-close-the-world-came-to-nuclear-disaster
Ehrlich, P. R. (1968). The population bomb. Ballantine Books.
https://www.theguardian.com/environment/climate-consensus-97-per-cent/2018/jan/01/on-its-hundredth-birthday-in-1959-edward-teller-warned-the-oil-industry-about-global-warming
Assume 60 billion have been born (this is Leslie's number), annual births are 120 million, and grow 2% a year: Cumulative future births after t years would be (120M/0.02)(e^0.02t − 1) = 6B(e^0.02t − 1), which can be solved algebraically for t=120 (50%/2085) and t=263 (95%/2230). If we use a more modern estimate of 109 billion, that moves our 50% to 2112 and 95% to 2257, but I think using the period's estimate helps you get into the mindset of the period better.
118 billion ever born, 8.3 billion alive, 132 million annual births
https://forecastingresearch.org/pdf/existential-risk-persuasion-tournament.pdf
https://files.cercomp.ufg.br/weby/up/410/o/Our_Final_Hour_%E2%80%93_A_Scientist's_Warning_._2003_._Martin_Rees_c%C3%B3pia.pdf
This is me copying from IRB who estimates that the birth rate was 80 per 1000 in 1 AD, but went down to 60 per 1000 in 1200 AD.