Non-orthogonality implies uncontrollable superintelligence

by Stuart_Armstrong1 min read30th Apr 201247 comments


Personal Blog

Just a minor thought connected with the orthogonality thesis: if you claim that any superintelligence will inevitably converge to some true code of morality, then you are also claiming that no measures can be taken by its creators to prevent this convergence. In other words, the superintelligence will be uncontrollable.