Posts

Sorted by New

Wiki Contributions

Comments

This is my point exactly - "At most, climate change might lead to the collapse of civilization, but only because civilizations are quite capable of collapsing from their own internal dynamics"

Pessimistic view of climate change I get from the fact that they aimed at 1.5C, then at 2C, now if i remember right there's no estimation and also no solution, or is there? 

In short mild or not,  global warming is happening, and since civs on certain stage tend to self-destruct from small nudges - you said it yourself, but it doesn't matter where the nudge comes from. 

2nd half I liked more than the first. I think that AGI should not be mentioned in it - we do well enough by ourselves destroying ourselves and the habitat. By Occam's razor thing AGI could serve as illustrational example of how we do it exactly.... But we do waaay less elegant.

For me it's simple - either AGI emerges and takes control from us in ~10y or we are all dead in ~10y.

I believe that probability of some mind that comprehended and absorbed our cultures and histories and morals and ethics - chance of this mind becoming "unaligned" and behaving like one of those evil and nasty and stupid characters from our books and movies and plays he grew up reading... Dunno, it should be really small, no? Even if probability is 0.5, or even 0.9 - still we got 10% chance to survive...

With humans behind the wheel our chance is 0%. They can't organize themselves to reduce insulating gases emissions! They can't contain virus outbreaks! If covid was a bit deadlier - we'd be all dead by now...

I mean I can imagine some alien civ that is extremely evil and nasty would create AGI that initially would be also evil and nasty and kills them all... But such civs exist only in Tolkien books.

And I can imagine apes trying to solve human alignment in anticipation of humans arriving soon)))) Actually bingo! Solving AGI alignment - it could be a good candidate for one of those jobs for unemployed 50% to keep'em busy.

I don't understand one thing about alignment troubles. I'm sure this has been answered long time ago, but if you could you explain:

Why are we worrying about AGI destroying humanity, when we ourselves are long past the point of no return towards self-destruction? Isn't it obvious that we have 10, maximum 20 years left till water rises and crises hit economy and overgrown beast (that is humanity) collapses? Looking at how governments and entities of power are epically failing even to try make it seem that they are doing something about it - I am sure it's either AGI takes power or we are all dead in 20 years.

Answer by Vugluscr VarcharkaAug 30, 202120

In any scenario there will be these two activities undertaken by the DEF ai:

  1. Preparing infrastructure for its initial deployment: ensuring global internet coverage (SpaceX SATs), arranging computing facilities (clouds), creating unfalsifiable memory storages etc.
  2. Making itself invincible: I cherish hope for some elegant solution here, like entangling itself with our financial system. Using Blockchain for them memory banks.