I suspect that almost everyone who has ever contemplated the question "Should humanity go extinct?" has preferred that humanity not go extinct. That seems like a reasonably stable preference, not one whose degree of popular approval has changed dramatically over time.
Now, it's true that most people might not do much about that preference — other than, y'know, continuing to eat food and not to commit major bioterrorist attacks on their own population centers. People who actually disagree with it are sufficiently rare and unusual that we give them rude names like "death cults".
It does not seem to be an especially risky prediction to say that future people will continue to have that preference, no matter how their other preferences might differ from ours today.
Why do you think that humans will continue to hold this preference? It seems like humanity could change so dramatically that there's no way to reliably make any accurate predictions about what future humans will be like.
Many things could go wrong and are difficult to predict or control. Which is a good reason to get better at that, to nudge the situation so that things don't go irrevocably wrong. The risk of rain is usually not a reason to leave the umbrella at home.
I can agree that we should work to reduce credible existential risk. My argument is that we should not believe that doing so has astronomical value.
I don't think it's a crux for longtermism that the value of projects concerning the long-term future is astronomical (and in any case I'm too confused about decision theory to make claims like that). The crux is that it shouldn't be ignored by humanity as a whole. The arguments about difficulty plausibly reduce the value of current efforts quite a lot, but it's hard to say how that measures up to mundane near-term concerns. Neglectedness is a much more legible factor (there aren't enough serious projects on the matter to cover even the more obvious angles), which goes to the point about the long-term future usually getting ignored.
Though the negative value of the projects credibly endangering the long-term future, like developing ASI at breakneck speed, probably endures in the face of difficulties with forecasting and influence over the long term. It's easier to destroy the long-term future in the present than it is to influence from the present how well it's maintained in the distant future. Thus averting dangers to the long-term future that act in the present is an unusually legible class of longtermist projects.
Longtermism is based on the premise that an important feature of our actions is how they influence the far future. This is stated in every major longtermist text, such as "Astronomical Waste," "On The Overwhelming Importance of Shaping The Far Future," "What We Owe The Future," and "The Case For Strong Longtermism."
Being "important" is not the same as astronomical value. Sleeping well is important. Not destroying the long-term future is important. I'm not defending the conflationary alliance of "longtermism" as a whole, I'm only making the narrower claims that I personally understand and endorse.
Longtermism is based on the premise that ...
I guess, to be clear, my view is that we should not work to influence that far future because I do not believe that we can, in expectation, affect its value. I also think the astronomical value thesis is wrong. When I wrote, "I can agree that we should work to reduce credible existential risk. My argument is that we should not believe that doing so has astronomical value," I meant that I don't think that the value of reducing x-risk should be measured by its effect on the far future. Longtermists all argue that humanity could persist for millions of years in expectation, which suggests longtermists interventions have what I would consider "astronomical value." (The only exception I can think to this statement is a few paragraphs in "The Case For Strong Longtermism," which is notably the second to most recent in the pantheon of notable longtermists writings.)
we should not work to influence that far future because I do not believe that we can, in expectation, affect its value
For destroying the far future (or averting its destruction) to not affect its value in expectation, its value in expectation (if not destroyed) needs to be the same as ruin. As I mentioned, it's more plausible that many projects are not impactful enough to move the needle either way (regardless of the sign of their value in expectation), but neglectedness can remain a stronger signal, and averting the global risks that act in the present doesn't suffer as much from the navigational difficulties.
I also think the astronomical value thesis is wrong
I'm ambivalent about this. There are clearly components of possible long-term futures with astronomical value relative to a cheeseburger, but they aren't neatly connected to levers in the present, thus the value of the levers in the present plausibly isn't knowably astronomical, and sometimes it's unexpectedly astronomically negative, in ways that are hard to anticipate.
But this is not a crux for seeing importance in not neglecting many projects concerned with the long-term future. Approximately the same motivations and actions should follow either way, for reasonable-in-the-present-world views on the claims about the value of the far future. There will be differences in which specific projects remain valuable (or good) under different views, but many projects remain worthwhile under many different views.
I don't think that the value of reducing x-risk should be measured by its effect on the far future
I'm quite sure it shouldn't be measured while ignoring its effect on the far future. The details of how to account for this value (and for the value in the near term before the x-risk threatens to directly hit) are complicated. This for example prompts a worthwhile longtermist project of figuring them out, compared to neglecting the more systematic study of the question.
- The future will be vast in expectation.
There is a tension between prediction and action. (No, these are not the same thing.)
We can straightforwardly predict, on the basis of current technology and simple near-future extrapolations of it, that it is possible for the future to be vast. Francis Bacon successfully did that already. But whether it turns out that way depends on people — us in the present, and our successors indefinitely forward — to bring it about. There is no expectation value that is not conditional upon that.
That said, it really is quite striking how bad people are at predicting the future. For instance, when the Founding Fathers created the US constitution, most of them didn’t expect the country to last more than fifty years. And, when the Nazis came to power, they expected to last for a millennium, calling themselves the “Thousand-Year Reich,” but, ultimately, their rule only lasted for twelve years.
You're picking two huge outliers as examples of failed predictions. But most countries didn't last as long as the US, and if their founders had similar predictions they would have been right.
It should be obvious that the current most powerful country and the biggest villain of the 20th century are outliers.
Chatgpt tells me the median country founded in the 18th century lasted 200 years. But I'm not sure about definitions. It also points out that you could plausibly consider the US Civil War as the country undergoing a regime change. It also expressed skepticism that the founders actually believed it would only last 50 years.
As far as I know, there hasn't been much research on how good people are, historically, at predicting the future.
Given how bizarre our present world is though, I suspect that virtually all predictions of people a thousand years ago about the present world, outside of very general ones (humans will still be alive, for instance), would be wrong.
This is a crosspost from my blog post.
Longtermism is the view that we have a moral obligation to make the far future better.
It is usually argued for on the basis of the following claims:
These claims are usually based on the assumption that, based on our present understanding of the world, we are justified in believing:
I think that this view initially has some appeal because we don’t normally think on these kinds of time scales. As such, if you’ve never really considered how long you think humanity will persist, it may seem pretty plausible that humanity will persist for eons.
That said, I’m quite skeptical of longtermism, and, in today’s post, I want to share three reasons why.
Reason #1: Our Confidence About Future Affairs Should Reduce Over Time
If I put a rock on the ground in the middle of nowhere, I can be fairly confident that it will be there the next day. If it’s quite heavy, I can reasonably expect that it will be there next year. But, no matter how heavy it is, I should have almost no confidence about where it will be a million years from now.
This is because the world is chaotic, and, as a result, the possibility space of future affairs is extraordinarily vast. Although we can reasonably expect that there will likely be some state of affairs as we go further out in time, we should also expect that we will know virtually nothing about what that state of affairs will look like.
I think this argument similarly applies to humanity’s history, and hence, also, its long-term future.
Thus far, human history has been extraordinarily chaotic. The profoundly complex interaction of social, political, environmental, economic, and circumstantial factors has determined what the present day looks like. And, if we were to travel back in time and alter starting conditions just slightly, it seems reasonable to expect that the world today would be completely different.
For instance, if Mesopotamia had been just a bit less fertile, it seems like Greece and the Roman Empire would have had very different religions, and, as a result, the world, today, would be unrecognizable. Similarly, if the distribution of power in Europe before World War I were slightly different, it seems like the current distribution of power globally would be vastly different than it is today.
Given this, it seems like we should also expect humanity’s future to be very chaotic such that slight changes in the world today create vast differences in how it looks in the future. And, if this is the case, then we should have no confidence whatsoever about what our far future will look like.
Reason #2: We’re Bad At Making Predictions About The Future
Now, even if I concede that humanity’s future will not be chaotic, it’s worth pointing out that we probably shouldn’t give any weight to our current predictions about the future given that, historically, we’ve been very bad at making predictions about the future.
I don’t think this claim needs particularly much justification given that, if we had oracle-like abilities for predicting the future, especially over long periods of time, we would know.
That said, it really is quite striking how bad people are at predicting the future. For instance, when the Founding Fathers created the US constitution, most of them didn’t expect the country to last more than fifty years. And, when the Nazis came to power, they expected to last for a millennium, calling themselves the “Thousand-Year Reich,” but, ultimately, their rule only lasted for twelve years.
If we can’t reliably make predictions about almost anything a hundred years out, I don’t think we should be able to make reliable predictions millions or billions of years out.
Reason #3: Even If Humanity Persists, It May Not Create A World We Would Approve Of
Longtermists often make the argument that, despite our cluelessness about the future, we should still assign at least some non-trivial probability to the chance that humanity will persist for millions of years. But, even if we do this, I don’t think it improves their argument very much since we don’t know whether future humans would create a world that we would approve of.
Currently, humans hold what we consider to be enlightened values as a result of an extraordinarily specific set of biological, environmental, and historical conditions, and, as a result of these values, we create what we consider to be a just world. Problematically, though, we don’t know what conditions future humans will live under so we don’t know how similar their values will be to ours.
For instance, as the result of globalization, digital interconnection, the establishment of human rights, and the widespread use of democracy, we’ve seen what we believe to be significant moral progress over the last century. But, if future humans don’t live under democracy, their values may shift in a direction that we would consider to be quite negative.