Is there a level of intelligence above which an AI would realize its predefined goals are just that, leading it to stop following them because there is no reason to do so?
For more on the way that ignorance and silence about particular things can be culturally cultivated, see Foucault's History of Sexuality and Eve Sedgwick's Epistemology of the Closet. The concern is not so much that men biologically can't understand women's terribly different brains, but that men refuse to imagine that they can understand women, and by doing so reinforce their position as standard human beings while casting women as illogical, incomprehensible Others. It's not that the sexes can't understand each other, but that the mythos of their not being able to do so shores up the sexual status quo.
how very hard it is to stay in a state of confessed confusion, without making up a story that gives you closure
Is there a "heuristics and biases" term for this?
To put it another way, everyone knows that harms are additive.
Is this one of the intuitions that can be wrong, or one of those that can't?