A Butlerian genie or AI Butler is an AI that understands material objects and technology extremely well, but has been somehow averted from modeling human minds, or possibly its own mind, overly well.
This is because many of the dangers in AI seem to be associated with the AI having a sufficiently advanced model of human minds or itself, including:
This is a very severe restriction and might be overly difficult to do coherently.
However, if we could in fact have an AI with an excellent grasp of how to accomplish material goals not involving the direct modeling and manipulation of human minds, it would not be irrelevant; it could carry out instructions that were game-changing for the value achievement dilemma.
While a Butlerian AI would still require most of genie theory and corrigibility to be solved, it's plausible that the restriction away from modeling humans, programmers, and some types of reflectivity, would collectively make it significantly easier to make a safe form of this genie.
Thus, a Butlerian AI is one of few open candidates for "AI that is restricted in a way that actually makes it safer to build, without being irrelevant".
The term 'Butlerian AI' comes from Frank Herbert's Dune, in which the Orange Catholic Bible contains the injunction, "Thou shalt not make a machine in the likeness of a human mind."