Currently, in most fields, the foremost experts are humans. They form a culture that pushes on the envelope of knowledge, uncovering facts and developing methods. Already, machine intelligence is able to uncover facts, and increasingly develop methods. But those facts still by and large are integrated into human knowledge through...
In the forgotten past, the rules of the game-world were set. Many of them accidentally, as arbitrary forms - simple enough to produce interesting emergent structure, but still arbitrary - but many of them with clear structure in themselves, signifying intent. The intent and the reasons are long lost in...
Some ideas how to set up the AGI training such that it wouldn't pose existential threat to humanity, even if its ability to manipulate us and (indirectly) our surroundings were superior to us. If these ideas have already been elaborated and/or debunked elsewhere, please let me know! Different ways to...