This is a test chapter of the project described here. When the project is completed in its entirety, this chapter will most likely be the second one.

We want to create a sequence that a fifteen or sixteen year old smart school student can read and that can encourage them to go into alignment. Optimization targets of this project are:

It must be correct. Obviously.
It should not be boring. It should be inspiring instead.
It should be easy to read. It should not be hard to read for school students.
It should not be stylistically similar to other texts with the same function. It should not be redundant.
It should contain as few technical details as possible. It is not a "how neural networks work" textbook.
It should help the reader enter the discourse of AI Safety and perceive other relevant texts in its context. It is more like "how to understand AISafetyspeak" textbook.
It should contain as little information which is likely to become obsolete in the next four to five years as possible. Because it's for school students.

All images are generated by StableDiffusion.

Most likely, this is not the final version of the text. We welcome comments and criticism, both about the goals set and about the tactics for achieving them. However, please explicitly separate the former from the latter.

Superhuman Artificial Intelligence is dangerous to us. To be precise, it is going to be a real threat to the existence of the human race, as soon as we create it.

Why? Because we still have no idea how to make AI share and obey our ethics as it makes its decisions.

You might say, "Well, that smart AI is gonna be Really Smart so it can understand our values!"

*AI is so smart, and consequently so kind, digital illustration*

But there's

one tiny issue:

to understand the values and objectives doesn't mean to accept them as your own.

If people are capable - and they are, you can read Machiavelli or pop in some good advertising agency - of achieving their objectives, using, but not sharing the others', then what stops self-educating AI to learn it from us?

We're so disappointed and angry with politicians when we realize that their election promises are nothing but a means to promote themselves, not to make the world a better place. So how would it feel when we'd find ourselves fully controlled by superhuman intelligence and figure out we've been manipulated - but only in the moment when...

Posts

Wikitag Contributions

Comments