leogao's Shortform

leogao

This is a special post for quick takes by leogao. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

I think it would be really bad for humanity to rush to build superintelligence before we solve the difficult problem of how to make it safe. But also I think it would be a horrible tragedy if humanity never ever built superintelligence. I hope we figure out how to thread this needle with wisdom.

I think the same world that coordinated well enough to do a centuries-long AGI pause (without heralding a dark age of negative economic growth or global totalitarianism, etc) is probably also more than capable of preventing thermonuclear war, extinction-level artificial pandemics, grey goo, etc.

At that point your biggest risks are natural x-risks (very low), authoritarian backsliding, meme wars, and some fraction of unknown unknowns.

1Canaletto1y

>slowing down for 1,000 years here in order to increase the chance of success by like 1 percentage point is totally worth it in expectation. Is it? What meaning of worth it is used here? If you put it on a vote, as an option, I expect it would lose. People don't care that much about happiness of distant future people.

4Stephen McAleese1y

In 2022 I wrote an article that is relevant to this question called How Do AI Timelines Affect Existential Risk? Here is the abstract: [...] Artificial Intelligence as a Positive and Negative Factor in Global Risk (Yudkowsky, 2008) is also relevant. Excerpt from the conclusion: [...]

leogao's Shortform

8

Ω 4

leogao's Shortform

8

Ω 4