I think there's something about how hypothetical and unprovable all of it is as well as the fact that it presents these overwhelming stakes that makes it kind of religious in nature.
Climate change activists and longtermists have similar concerns, but it feels like longtermism promotes a sort of fanaticism and mysticism that climate change does not.
I think intrinsically any prediction of the future not directly tied to things we have examples of from the past, such as climate cycles, will feel like this. And the future almost certainly will have things we don’t have lots of past evidence for. Maybe part of it might be that people just don’t actually try to think about or model the long term future very much, or the value in it, so the few people who do wind up being longtermists? Or maybe I’m just underexposed to other schools of thought. I wonder what other visions for the long term future exist, that depart radically from the longtermist point of view and are robust against scrutiny?
I generally think the long-term future is so chaotic that we shouldn't have any confidence in pretty much any model. I also am not very familiar with other models for the long-term future, other than those proposed by people who are concerned about s-risks, i.e. the fear of an infinite dictatorship.
I note you can disbelieve 2) and 3) (humanity may merely persist for another thousand years, and every day will be roughly like today) and still believe 1) (x risks are worth worrying about).
And even then, if everything stays the same, most value will still be in the future: 10 billion humans, turned over every generation - say 25 years - 400 billion total, vs the 100 billion that currently ever have lived. A 1% chance of x-risk reduction saves 4 billion people; hence, 1/4 billionth of a percent of permanent x risk reduction is worth at least $4k (price to save a life from malaria today).
If it’s not permanent, and the price has to be paid every generation, that’s still worth $4k per hundred millionth in this generation. Or $400 billion per generation to permanently avert 1%, or $16 billion per year. Times 5 for the price to save a median life, rather than from malaria; times 50 for population weighted mean; times 100 at wealthy country rates. Times 1 - (.025 x generations passed since today).
Even the Lindy effect - things can be expected to last about as long as they already have - applied to all of humanity still gives us another 100 billion humans, or a 1/4th multiplier on everything above.
Catastrophes/c-risks are subject to the same math. Averting ones that reduce population/QALYs by 100x by 1% forever just means multiplying 1/4 billionth by .99. Or averting that every generation: one one hundred millionth by .99.
You can play with the variables until you find a number you like, but I think unless you already think humanity will unavoidably go extinct very soon, or suffer a very large permanent reduction (such as by permanent deindustrialization), arguments to not spend significant resources on c/x risk don’t seem very convincing to me?
As for c/x risk itself. I’m pretty sure most people would agree we face at least 1% risk per generation. I think you’d certainly get that kind of answer during the Cold War, or from most climate change believers.
As for unfalsifiability. Species past and present suffer population crashes and extinctions all the time, especially during certain periods of history- like today. What makes humans any different? Our intelligence and coordination? So that will save us, for free, with no resources spent on the problem? Intelligence and coordination can’t or won’t be applied to making catastrophes and extinctions happen faster? The offense/defense balance for that heavily favors defense? These claims need arguing. Survival is not automatically a privileged hypothesis.
So. I think if you strip out the cosmic transhuman stuff - which, to be clear, I disagree (edit: that is, I think cosmic transhumanism is plausible) - “humanity lives for another 250 years roughly like today” essentially asserts history ends here and we are at the top of every S-curve relevant to sapient population or welfare, or that every rise on one curve is equally and unavoidably countered by a fall on another - I still think c/x risk should occupy the attention of very many people.
To be clear, I'm merely arguing that it should be acknowledged that longtermism has some quasi-religious aspects. If you want to see my criticisms of longtermism from an academic perspective, I've wrriten about that here.
I deny the astronomical value of x-risk reduction, but, if there are credible x-risks, I would be supportive of throwing enormous amounts of resources towards them, as the value is extraordinary even if humanity only persists for 75 more years. (I'm unsure how far out we should measure the value of x-risk reduction, but 75 years seems possibly reasonable and still yields enormous value. The issue with projecting out too far in my view is that we simply don't know whether future humans will create a world that we consider valuable or how many of them there will be. People will argue that we should assign some non-trivial probability to astronomical values, but I think we have deeper uncertainty than that.)
I don't have any opinions on AI x-risk, although I'm quite skeptical of biorisks, since concern about them seems to be a relatively fringe view within the biosecurity community.
That said, I think we both agree on that, if x-risk is high and tractable, it's extreme important, and that the cosmic transhumanist stuff is unreasonable.
Thanks for engaging with what I wrote.
I read your article. My thoughts:
Re #1: Your examples touch on culture and human organization, but not on physics or broad ecology. “There will probably be sapient life in the future” and “The most aggressive and robust forms of self replication will probably take over the universe” is more defensible than “Democracy will persist through eons.” We should be very unsure, yes, but that unsure?
#2: You don’t need to predict fine details. Again, your examples center around human organization - which is impossible to reason about on long timelines, yes. But “will there be sapient life” is a broader question, which swings around things like x-risks. Which you could argue are as hard to reason about as the details of human organization - and that would be worth checking!
#3: Yes, value drift is a concern, but outside of suffering-risks, where either we - or a plurality of feasible moral systems - think that our descendants would effectively be sentenced to eons of torture not worth surviving through, our ancestors have been generally content with value drift, if only because they couldn’t perceive it. At the very least, we value being alive and having children, and so does basically everything else on Earth, and it’s hard to see why it would drift, being that it’s so instrumental.
Re your comment:
Sorry if I wasn’t clear - I edited my comment to note that I think cosmic transhumanism is plausible.
I’m glad we agree x-risks are generally very important, if they exist : ) And thank you for responding!
I am curious where you heard that biorisk is fringe within biosecurity? I don’t know of a source that proves that, and would like to know. As for AI risk, the big AI Impacts surveys of ML researchers put that at 5%.
Re #1: I still think the future's so chaotic that the claim “The most aggressive and robust forms of self replication will probably take over the universe” is too strong. If you're familiar with conway's game of life, some starting conditions create enormous vast worlds, others blink out. Just by looking at a set of tiles, you couldn't really guess how long it would persist without studying conway's game for a while, but we're in a worse position because we can't even re-run the universe to see what happens. I could see the case for sapient-life re-emerging, assuming humans were to just go extinct right now from an asteroid or something, but I don't know how humans will affect the world so I don't know how likely such a scenario is.
Re #2: I think that past humans were bad at making long-term broad predictions too, which would suggest that we will be as well. I'm not sure about high likely x-risks are or how confident we should be in them, although I could imagine there being a very credible threat.
Re #3: If we're talking about the far future, I think we can make essentially no claim about the values of future humans will be because we know so little about the far future. But, if I were to entertain the idea that we can, I would say that it seems like "we value being alive and having children" could easily drift. For instance, many demographers expect the human population to start declining this century because we value having children so much less than we used to. This trend could intensify even more. Similarly, future humans may modify themselves such that their "valuing being alive" is very different from ours. For instance, future humans might want want to maximize their experience so they may be willing to trade-off on intensity of experience over duration of life.
In regards to biosecurity, I wrote this post, showing that we lack good data in terms of how concerned biosecurity experts are about biorisks. David Thorstad, a philosopher who worked at GPI, has stated that he believes it is a fringe concern. I asked a microbiologist concerned about biological warfare, and he said that most bioesecurity experts only very slightly concerned. I also asked a journalist who said that, in their experience, it seemed to be a very fringe concern. Also, I haven't looked into it very much, but it doesn't seem life federal governments take the risk of catastrophic engineered pandemics all that seriously, besides some relatively minor low-cost actions.
In regards to AI, I don't have much of an opinion currently. That said, if I had never read anything about AI and knew nothing about the field's history, and I learned that 30% of ML researchers thought there was a 5% chance of AI killing everyone, then I would think that enormous amounts of resources should be poured into ensuring that this does not happen, assuming that there was nothing significantly biasing ML researchers views.
This is a cross post from my blog post.
Longtermism is typically framed as an academic philosophy or as a pet project of billionaires, but it’s worth pointing out that, in many ways, it also functions similarly to a secular religion.
Although I do think that longtermism should be attacked and defended on academic grounds, I also think it’s important to take seriously the ways in which the view has a religious appeal.
This will not only allow us to think more clearly about it as a philosophy, but it will also help those who are questioning their support of it to better deconstruct the view.
With that said, here’s five ways longtermism is like a religion:
1. Longtermism makes unfalsifiable claims about things that cannot be known.
Religions often make claims about the nature of reality that cannot be disproven. Since there’s no way to challenge them, their followers can maintain faith regardless of what counterarguments they hear.
Similarly, longtermists make a lot of unfalsifiable claims about what the future will look like, despite the fact that we know very little about what the future will hold.
Longtermists variously believe that:
Each of these claims may appear plausible on the surface, but it’s important to point out that we have very little information on which to assess them and no ability to falsify them.
2. Longtermists make the unknowable vivid and concrete.
Religions often make claims about things that cannot be known and then go into great detail about the nature and quality of these things. This makes them feel real and grants them more importance.
Longtermists do the same with the far future.
Although we have no clue what the far future will be like, longtermists describe it in such vivid terms that it feels like a place that we could inhabit.
For instance, in What We Owe The Future, William MacAskill writes, “Consider the very best moments in your life—moments of joy, beauty, and energy, like falling in love, or achieving a lifelong goal, or having some creative insight. These moments provide proof of what is possible: we know that life can be at least as good as it is then. But they also show us a direction in which our lives can move, leading somewhere we have yet to go. If my best days can be hundreds of times better than my typically pleasant but humdrum life, then perhaps the best days of those in the future can be hundreds of times better again.”
3. Longtermists believe their actions have astronomical stakes
Religious individuals often believe that they are privy to special knowledge, and that, as a result, their actions are enormously consequential.
For instance, many Christians believe that, if they convert someone to Christianity, they’re not merely convincing someone to follow their religious figure, but, instead, that they’re literally saving someone from eternal torture.
Longtermists, similarly, believe that their actions are uniquely important.
Because they expect the future to persist on a timescale of billions or even trillions of years, they believe that, even if they reduce existential risk by an extremely small amount, they are effectively saving vast amounts of people. In Nick Bostrom’s paper “Astronomical Waste,” he even suggests that by reducing absolute existential risk by one percentage point, one would, in expectation, save 10^36 human lives.
4. Longtermists devalue the present world.
Religious people often believe that the present world is a shadow of the world that will eventually come and that, as a result, the present world isn’t particularly important.
Longtermists similarly believe that virtually all value lies in the far future, and that, as such, the present world isn’t particularly important.
For instance, longtermism’s strongest proponents, Toby Ord and William MacAskill, were originally founders of the effective altruism movement, in which they encouraged people to donate significant portions of their income to help save the lives of people in low-income countries.
But, now, because longtermism suggests that the present world has minimal value, Ord and MacAskill generally promote working on longtermist interventions such as pandemic preparedness and AI safety instead.
The lives of those who are living today are seen as almost insignificant compared to what humanity will one day achieve, and that, as such, humanity’s current problems aren’t worth being concerned about.
5. Longtermists prophesize the end of the world.
Religious people often believe that the world is likely to end within their own lifetime. This gives their actions unique importance and causes them to view the world on a more cosmic scale.
Longtermists, similarly, believe that humanity is currently going through a period of heightened existential risk, during which virtually all of humanity’s future value could be destroyed. One of the most prominent longtermists, Toby Ord, suggests that there is a one in six chance that this could happen this century.