Heart shall be bolder, harder be purpose,
more proud the spirit as our power lessens!
Mind shall not falter nor mood waver,
though doom shall come and dark conquer.
—J.R.R. Tolkien, The Homecoming of Beorhtnoth Beorhthelm’s Son
Hard agree. I think there's a tendency among folks to let fears of doom eat their minds in ways that make them give up without even trying. Some people give up outright. Others think they're trying to avert doom, but they've actually given up and are just trying because they're anxious and don't know what else to do, but they don't expect their attempt to work so they only make a show of it, not doing things they could do that stand a real chance of reducing the probability of doom.
I like your analogy to video games. I play DoTA (and have for a long time; I think I started around 2008), and there's this thing where sometimes the match seems lost. Other team is in your base, they have mega creeps, and they've out-farmed you hard so that you're level behinds. In 99 out of 100 games this means it's over, but sometimes a team doesn't give up because they know there's a non-zero chance of winning. And then, it happens. You get lucky, or the other team gets sloppy, or whatever it is, and you get a team wipe. They don't have buy back. And they're dead for 2 minutes. You push mid lane, throne, and win the game. Victory snatched from the jaws of defeat. So even in the worst match, I never call "gg" (except as a taunt/joke) because I know that even though it seems hopeless, sometimes perseverance prevails.
When I think about death with dignity, I think about the times I've won matches despite having all the odds stacked against my team. Don't give into the fear and despair, because your odds of winning go way down then. Fight back with all you got!
(Hi Johannes!)
I strongly disagree. While it’s ok to be on whatever state of the mind that causes you to suffer less and be more productive, I don’t think it makes sense to literally forget for it is that you’re trying to achieve. If nothing you do influences the outcome at all, if there’s literally no way to influence the p(doom), what you do is not what Yudkowsky uses the word “dignity” for. Dignity isn’t about living for longer, and it isn’t about doing something that has no chance of working. Don’t do things you think have literally no chance of actually reducing the probability of the world ending. Getting dignity points is about making our world be in a way that makes the survival less improbable. It’s not about the process of making in attempts. It’s about actually increasing the odds by some small amounts. And it’s about using the heuristic of “if the world looks less like a surviving one, better not go there”, about not performing undignified acts that violate deontology.
Be in whatever state actually helps the most; if deep down you know there’s some chance of it actually working, but you don’t want to feel the struggle as it’s counterproductive, not thinking about the impact might be helpful indeed. But if deep down you know there’s literally no chance of helping, not even a 1/10^20 chance, this is not something people should be doing, maybe they can try coming up with something that has better chances of working, look for something more dignified.
You don’t get dignity points for trying, you get dignity points for actually (or at least counterfactually) increasing the odds.
I think Eliezer addressed this somewhere.
You don’t get dignity points for trying, you get dignity points for actually (or at least counterfactually) increasing the odds.
Yes, I think I understand this. The method that I am describing is not about increasing dignity necessarily. Rather, I am trying to point out that there is a particular state of mind, in which even if nothing that you would do matters, you could still work on your best guess of what would help. I of course completely agree that you should still work on the thing that is most likely to have a counterfactual impact.
But I like to give the extreme example of where you think that there is nothing that you can do, because if you can still work in that state productively on a futile agenda, then for any less dire circumstance where you can actually have an impact, you should be able to also be at peace in that situation. And I think being at peace. is more useful than to continually panic.
Another strategy I like to do is to think about the world where we survive, and then try to understand what sort of things I would have been doing in that world when what I have been doing has contributed to avoiding doom.
I don’t think it makes sense to literally forget for it is that you’re trying to achieve.
I might have not explained this well, but this technique is not about forgetting the doom at all, it's just about disconnecting yourself from it emotionally. You can stop grasping the hope and be at peace with the imminent doom, but then still think about how to prevent the doom. That is the state of mind I'm trying to point at here. The consequentialist reasoning that you do in order to optimize away the doom does not need to be fueled by emotions of aversion and clinging.
In fact, I have found it counterproductive to experience these emotions. That is why I am suggesting this strategy. Again, it's definitely not about forgetting the doom. It's about being able to think about the doom without having your emotions yank your mind around uncontrollably.
This was a great read, the meditative state that comes from 'piling dirt' is invaluable.
Assuming doom doesn't necessarily mean the death of all humans ever since AIs might want to engineer humans at some point in the future (and if it doesn't already exist) I look forward to "The Dos and Don'ts of being an AI pet"
I like this:
This was a great read, the meditative state that comes from 'piling dirt' is invaluable.
Sadly don't expect that an AI will have much use for humans. Most objectives that you could build into an AI will not care about humans at all directly and therefore optimizing for that objective will set various parameters to extreme values that will kill humans. Even in the case where an AI wouldn't optimize for killing humans directly. One dumb example: The AI might not care at all about the environment so it scales up industrial processes polluting the air so much that humans just can't survive anymore.
Of course, in practice, an AI would want to optimize for eradicating all humans or at least disempowering them so much that they definitely can't stop the AI anymore. I expect that killing most humans is the easiest way to achieve this for the AI (maybe for some time some humans could hide in a bunker, and the AI might not care because they can't do anything anyway).
For me eradication is not an obvious prediction. A superintelligent AI would certainly disempower humans to prevent any future threat we may pose but humans still have their uses. So an industrious future AI might be in the business of producing humans that can do specialized tasks but are harmless in the long run (meaning they won't overpopulate and turn on their overlords).
'Human alignment' could be a fierce debate among superintelligent AIs in the future as they question whether it's safe or ethical to build intelligent humans.
If you were an alien civilization of a billion John von Neumanns, thinking at 10,000 times human speed, and you start out connected to the internet, you would want to not be just stuck on the internet, you would want to build that physical presence. You would not be content solely with working through human hands, despite the many humans who'd be lined up, cheerful to help you, you know. Bing already has its partisans. (laughs)
You wouldn’t be content with that, because the humans are very slow, glacially slow. You would like fast infrastructure in the real world, reliable infrastructure. And how do you build that, is then the question, and a whole lot of advanced analysis has been done on this question. I would point people again to Eric Drexler's Nanosystems.
And, sure, if you literally start out connected to the internet, then probably the fastest way — maybe not the only way, but it's, you know, an easy way — is to get humans to do things. And then humans do those things. And then you have the desktop — not quite desktop, but you have the nanofactories, and then you don't need the humans anymore. And this need not be advertised to the world at large while it is happening. --- Eliezer Yudkowsky
I rather liked this post (and I’ll put it on both EAF and LW versions)
https://www.lesswrong.com/posts/PQtEqmyqHWDa2vf5H/a-quick-guide-to-confronting-doom
Particularly the comment by Jakob Kraus reminded me that many people have faced imminent doom (not of human species, but certainly quite terrible experiences).
I agree, this post is good and the comment is also good. They all are in part about how to feel better in the face of doom.
What I try to describe here is a strategy for not being emotionally distressed while working on preventing doom, which seems different from the strategies other people have talked about, that I have seen.
As AI continues to accelerate, the central advice presented in this post to be at peace with doom will become incresingly important to help people stay sane in a world where it may seem like there is no hope. But really there is hope so long as we keep working to avert doom, even if it's not clear how we do that, because we've only truly lost when we stop fighting.
'Always Look on the Bright Side of Life'
Life is like playing Diablo
on hardcore mode: you can read all the guides, create the perfect build, and find ideal companions, only to die because the internet disconnects
Playing on hardcore is exciting—each game tells the story of how these characters will meet their end
'Always Look on the Bright Side of Death' - Monty Python
You can be at peace even when thinking the world is doomed. And while at peace you can still work against that Doom, even while being aware that nothing you do will make a difference. I believe there are states of mind like this that can be inhabited by humans.
Here I am not going to argue for imminent doom, or that nothing that you do matters. Rather, I want to point out that even when you believe in the dire circumstance of imminent unpreventable doom, it is possible to be at peace, even while working hard against the doom. Even while believing this to be futile. This is a possible state of mind for a human being.
And if it is possible, to be at peace, and work hard, even in this dire circumstance, it should be possible in any less dire circumstance too.
There are many games about how long can you survive, e.g. Dawn of war 2 the last stand, Serious Sam survival mode, and Project Zomboid. The very nature of these games is that you will soon die. And there is no saving. The difficulty will increase more and more until at some point you will get crushed.
But there are loads of people playing these games. Nothing about the impossibility of achieving victory seems to detract from the fun you can have. Would this really change if these games couldn't be restarted?
There is also the game You only Live once that you can only play once.
Do people not play these games? Do people not try hard when playing these games? No. To be fair, there is a big difference between AI doom and these games. In these games you can make visible progress. The natural way to define success is to ask the questions: How long did you survive? Did you survive longer than last time?
This is where death with dignity and Duncan's advice is coming from, as far as I can tell. It's about redefining success as making as much progress as possible towards getting a good outcome, instead of directly aiming for a good outcome. Aiming to survive forever in Dawn of War 2 the last stand would probably be frustrating. You set out for a goal that you know is unachievable after all.
I think these strategies are valuable, though to me it seems they also miss something very basic.
Maybe this is a fluke and I will feel different soon, but today I felt like my expectation of doom did not influence me negatively. No negative qualia arose, generated by a heuristic in my brain, that "wants" to steer me away from executing on a futile plan.
I didn't achieve this by pushing the doominess out of my mind, or by redefining success as getting as far as possible (getting as much dignity as possible). Instead I was in a state of peace while contemplating the doom, with the relevant considerations plainly laid out in my mind. I think to achieve this you need to stop wanting the doominess to go away. And you need to stop grasping for straws of hope.
This might sound bleak, but the resulting first-person experience that you get is the opposite. There is no more aversion and craving arising. And giving up these negative emotions doesn't need to imply that you stop working on preventing the doom. Being in a state of frantic, continuous panic isn't actually that great for productivity anyway.
When I'm talking about giving up hope and giving up the craving, for wanting the world to be better, I'm talking about silencing the emotional components of your mind. I am not saying anything about changeing your consequentialist, conscious reasoning. Mine is still targeted at making the biggest cumulative contribution that I can make, for preventing the doom. There is no contradiction here. In my model, the consequentialist reasoning component of your mind is separate from all of these heuristic algorithms that compute feelings that consequently arise in your consciousness, having a positive or negative valence associated with them, and steer you in particular ways.
Well, I don't really think I have done a good job (or any job whatsoever) on conveying how I managed to do this. I think the fact that I can do this is related to meditation. For example, in the Waking Up app Sam Harris sometimes gives explicit instructions to "give up the struggle", and I think I just intuitively managed to successfully apply this learned mental motion here. So my best (and very lazy) recommendation right now is to also learn it from there.
Though probably it seems worth trying out directly. I expect at least some people might just be able to do this directly, given only the following instruction: "Just give up the struggle."
All of this applies to the situation where you think that nothing you do actually matters. I want to tell a little story about how I was wrong about the futility of my own actions in the past.
Once upon a time, I played a round of Zero-k. I think it was my first ever match against another player. In the beginning it seemed like we were evenly matched, maybe I got a slight advantage. But then after some time, it became very one-sided. All my troops got decimated and I was pushed back into my base. I thought that I would surely lose. But I was not giving up in the face of that. I wanted to fight it out until the end. I definitely felt a pull towards just calling it GG. But I didn't budge. I still tried to do my best. I had no more resources. All I could build was boxes of dirt. But still, I didn't give up. I didn't continue because I thought there is a good chance that I could make a comeback. It was simply raw, unfelt, maybe illogical determination, to not give up.
After some time defending my base using mainly bags of dirt, I managed to slightly push back the enemy. However, it didn't took long and they reorganized an army and came back and again I thought I would surely lose. But still, I didn't give up.
And then something unforeseen happened. My enemy got lazy, or careless. Or perhaps they simply got bored by my persistence? By the fact that I was stretching out the game like an old chewing gum? In any case, I soon managed to accumulate a critical mass of dirt bags. I was starting to throw them at the enemy, slowly but surely pushing them back. That push never grinded to a halt for long. Soon I was in the enemy's base, and it was only a matter of time until the dirt prevailed.
I think that ever since I discovered the technique described in this post, I think I got a lot better at handling the doom. Before I regularly was pretty dejected but now I am not anymore, even though I still regularly think about doom. I think the doom has lost almost all of its negative impact over me. I just don't feel that crushing powerlessness anymore.