Since AI safety has become incredibly popular recently, I quickly finished the draft I’d been working on for a long time and am finally publishing a comprehensive FAQ on AI risks. It’s also available on my website in a more polished version, where it will be updated regularly.

It’s intended for people who are new to the field after seeing all this news and posts on X, and who don’t understand why AI would even want to kill us. That said, people who are already familiar with the topic might also find a couple of interesting points.

I was in a real hurry to finish this text and wanted to make it understandable to as wide an audience as possible, so I apologize in advance if you find any serious mistakes. Please let me know about them, and I’ll correct them. Although I wrote the text by hand (without AI), I used AI to translate it into English (since it’s not my native language and I was in a hurry while the topic is still trending among newcomers). Please let me know if you find any parts that were translated incorrectly.

AS SHORT AS POSSIBLE:

AI continues to develop, and in the future it may be capable of killing every living thing on this planet. It does not need consciousness or humanoid robots to do this, as shown in films. Our typical ideas about ‘evil’ AI prevent us from seeing the real sources of the problem. Misalignment with human values is the main danger, and researchers believe that solving this problem is incredibly difficult. Progress in safety is not keeping pace with progress in AI capabilities. We need to take serious measures right now, as approving and implementing them could take years.

Not convinced? Read the objections ↓

PART 1. ‘I HAVE JUST LEARNED ABOUT AI RISKS’.

1. It is not intelligence at all.

You can call it whatever you like if you dislike the word ‘intelligence’. You can call it a system that can generalize, plan, write code, use tools, find information, operate interfaces, and carry out tasks. It just takes too long to say or write all of that. It is much easier to use the word ‘intelligence’.

1.1. It just predicts the next word!^[1]

The key word in this sentence is ‘just’. It downplays an actual capability by explaining its underlying mechanism. This is rather like saying, ‘The brain just transmits electrochemical signals.’ But that does not make the brain’s capabilities disappear. It is also important not to underestimate predictio...

H. Ryan Jones

H. Ryan Jones