Following is a brief of some parts of this paper on Aligning AI with shared human values.
The "why" behind most human actions is a universal seeking of pleasure and aversion to pain, so it seems natural that morality should be focused on "the greatest good for the greatest number of people".
This is why Utilitarianism emerged as a key idea in human values- that we make moral decisions from the position of a benevolent disinterested spectator.
In the paper this is mathematically translated as “maximizing the expectation of the sum of everyone’s utility functions.”
A utility function maps various scenarios to a scalar representing the pleasure associated with them. For eg: Completing a project... (read more)