BTW, Bostrom wrote about such last hope: https://nickbostrom.com/papers/porosity.pdf
My understanding of this article as TL;DR: AI should take a possibility that it will meet alien AI in space seriously. There is a chance that such alien AIs will be Friendly to their own civilisations and thus will build relation with our AI based on how it treated humans.Therefore, preserving humans is net beneficial, as it costs almost nothing, but can be used as an argument for better relation with other advance AIs, located on Grabby alien distance of 1 billion years.
In short, counterfactual alien AIs are an argument for preserving humans.
`He proposed a nearly insane plan because it was his only one, and he kept going.
Gandalf had a plan around which he could rally people around. Eliezer sees no such plan for friendly AI.
Generally, the act of rallying people around a plan means that other possible plans get less attention.
Yeah, of course, I agree. But, as I said, I didn't mean that Eliezer did something wrong. I am just rooting for someone else from the AGI Safety community who has hope and who also honestly sees that things are getting terrible, and this is not the reason to give up but the reason to fight harder. If all people who have a plan think it is not good enough and translate this mood to the public, then the public will not believe them, and all people get more depressed than motivated to do something. I think it's worse than having an insane plan but giving people hope. It's not about the logical side, like, are there any instructions on what to do (Anthropic, Conjecture, Red Wood, etc. have some). It is about the passion and belief of people who propose plans.
I have watched the Bankless interview with Eliezer Yudkowsky, and along with "Death with Dignity," they give the impression that Eliezer gave up. It's not the critique of Eliezer somehow. It's been a long hard way, and I appreciate what he has done and is doing. There is a place for grief and desperation, but there is a time for rise again and fight.
I love "The Lord of The Rings" because it teaches us to deal with situations when hope is almost lost. So many characters, even the wisest, have been telling how bad the problem is. Nevertheless, Gandalf never gave up, even when there was only a slight glimpse of hope. He proposed a nearly insane plan because it was his only one, and he kept going. He didn't know how hobbits could get to Mount Doom, and there were arguments that this was impossible. But there is always a chance that new possibilities will arise when you keep going. And you need to believe what you are doing. It's rational to assess the situation honestly, but you cannot win without motivation and a positive mood when all the most valuable people end up in depression..
I think "We're all gonna die" is a necessary shake-up for the public. But personally, I also wanna Gandalf here - some person who will motivate the community to action and infect them with passion and belief. It is not late to act when we are still alive. When a quest stands upon the edge of a knife, plans which seemed reckless and doomed to fail are better than nothing. For example, Eliezer said that social levers are hopeless, but other proposes are hopeless too. We may need to reconsider the government track.
“It's like in the great stories, Mr. Frodo. The ones that really mattered. Full of darkness and danger they were. And sometimes you didn't want to know the end. Because how could the end be happy? How could the world go back to the way it was when so much bad had happened? But in the end, it’s only a passing thing, this shadow. Even darkness must pass. A new day will come. And when the sun shines it will shine out the clearer. Those were the stories that stayed with you. That meant something, even if you were too small to understand why. But I think, Mr. Frodo, I do understand. I know now. Folk in those stories had lots of chances of turning back, only they didn’t. They kept going, because they were holding on to something. That there is some good in this world, and it's worth fighting for.”