The reward function is already how well you manipulate humans
We have many, many parables about dangers arising from super intelligent or supremely capable AI that wreaks ruin by maximizing a simple reward function. "Make paperclips" seems to be a popular one[1]. I worry that we don't give a more realistic scenario deep consideration: the world that arises from super...
Oct 19, 202220