LESSWRONG
LW

JerkyTreats

Message

One Does Not Simply Replace the Humans

In this post we discuss the "second point of lethality" in the helpfully curated AGI Ruin post. > A cognitive system with sufficiently high cognitive powers, given any medium-bandwidth channel of causal influence, will not find it difficult to bootstrap to overpowering capabilities independent of human infrastructure. The example to...

Apr 6, 2023•9

JerkyTreats

JerkyTreats — LessWrong

JerkyTreats

Message

One Does Not Simply Replace the Humans

Apr 6, 2023•9

JerkyTreats

One Does Not Simply Replace the Humans

JerkyTreats

In this post we discuss the "second point of lethality" in the helpfully curated AGI Ruin post.

A cognitive system with sufficiently high cognitive powers, given any medium-bandwidth channel of causal influence, will not find it difficult to bootstrap to overpowering capabilities independent of human infrastructure.

The example to support this theory is:

[The AI agent] gets access to the Internet, emails some DNA sequences to any of the many many online firms that will take a DNA sequence in the email and ship you back proteins, and bribes/persuades some human who has no idea they're dealing with an AGI to mix proteins in a beaker, which then form a first-stage nanofactory which can build

... (read 955 more words →)