Bluegreen

• Possible, seems unlikely. Requires two AI with different alignments, requires benevolent AI to respond to that sort of threat. Also falls under the first point.

• Incredibly unlikely: an AI is not going to structure itself so it works to fulfill the inverse of its utility function as the result of a single bit flip.

This is a bit more broad than cryonics, but let's consider more specific possible causes of extreme torture. Here're the ones that occurred to me:

An AI runs or threatens to run torture simulations as a disincentive. This is entirely a manipulation technique and is instrumental to whatever goals it has, whether benevolent or neutral.

• The programmers may work specifically to prevent this. However -- MIRI's current stance is that it is safer to let the AI design a utility function for itself. I think that this is the most likely and least worrisome way tortu

... (read more)