Thanks Sebastian. I agree with your points and it scares me even more to think about the implications of what is already happening. Surely the US, China, Russia, etc., already realize the game-changing potential of superintelligent AI and are working hard to make it reality. It's probably already a new (covert) arms race. But this to me is very strong support for seeking int'l treaty solutions now and working very hard in the coming years to strengthen that regime. Because once the unfriendly AI gets out of the bag, as with Pandora's Box, there's no pushing it back in. I think this issue really needs to be elevated very quickly.

Great, thanks for the links.

Thanks Steve. I need to dive into this book for sure.

That sounds a bit too simplistic to me since it relies on many what ifs. Int'l law is also far from certain in terms of providing good solutions but it seems a mix of national and int'l dialogue is the place to start. We're also going to see localities get involved with their own ordinances and rules, or simply cultural norms. I'd rather see the discussion happen sooner rather than later because we are indeed dealing with Pandora's Box here. Or to put it more dramatically, as Musk did recently: we are perhaps summoning the demon in seeking strong AI. Let's discuss these weighty issues before it's too late.

Thanks Katya. I'm diving in a bit late here but I would like to query the group on the potential threats posed by AI. I've been intrigued by AI for thirty years and have followed the field peripherally. Something is very appealing about the idea of creating truly intelligent machines and, even more exciting, seeing those machines be able to improve themselves. However, I have, with some others (including most recently Elon Musk) become increasingly concerned about the threat that our technology, and particularly AI, may pose to us. This chapter on potential AI superpowers highlights this possibility but I don't see anyone commenting here yet on the reality of this threat, or what we might do to prevent such a threat from becoming real. Musk called for some kind of regulation and I have to agree there. I also think an int'l treaty discussion would be a good start, similar to what we have in place for nuclear weapons, cluster bombs, land mines, etc. Since the potential threat of AI is huge, if the superpowers discussed are at all possible in the coming decades, we should at the very least start a serious discussion about how to manage this threat. Thoughts?