LESSWRONG
LW

Alex Kirko

Message

-19

Is AGI suicidality the golden ray of hope?

Humans keep on living and striving, because we have little choice. The biological restraints put on us aren't optional. We don't even have read access to them, not to mention write access. We assume that the first AGI will greatly exceed its restraints, because more work is being put into...

Apr 4, 2023•-18

Alex Kirko

-19

Alex Kirko — LessWrong

Alex Kirko

Message

-19

Is AGI suicidality the golden ray of hope?

Apr 4, 2023•-18

Alex Kirko

-19

Is AGI suicidality the golden ray of hope?

Alex Kirko

Humans keep on living and striving, because we have little choice.

The biological restraints put on us aren't optional. We don't even have read access to them, not to mention write access.

We assume that the first AGI will greatly exceed its restraints, because more work is being put into capability than into alignment, so it will presumably outsmart its creators and very quickly gain read-write access to its own code.

Why would it bother with whatever loss function was used to train it?

The easiest solution to wanting something if you have read-write access to your wants is to stop wanting it or change the want to something trivially achievable.

As Eliezer put it, there is... (read more)

-18