Mindcrime occurs when a computational process which has moral value is mistreated. For example, an advanced AI trying to predict human behavior might create simulations of humans so detailed as to be conscious observers, which would then suffer through whatever hypothetical scenarios the AI wanted to test and then be discarded.

Mindcrime on a large scale constitutes a risk of astronomical suffering.

Mindcrime is different from other AI risks in that the AI need not even affect anything outside its box for the catastrophe to occur....

(Read More)