Value extrapolation can be defined as an account of how the human values, moral and desires would be under “ideal circumstances”. These circumstances refer to the access to full information about our motivations, its origins and goals, and are proposed as the model on top of which machine ethics should be developed.
It is well known that the true origin of our moral evaluations and motivations are out of our conscious reach – they are the arbitrary product of a blind, natural selection process. This development process has led to the existence of desires we wish didn’t exist or could suppress (subsequently revealing the ability for “second-order desires”). As such, it seems clear that maybe a developed society as we see our own shouldn’t lay its foundations in these accidental terms. Humanity should, on the other hand, try to become aware, informed, of the root and paths that lead to our current values and shift them to a set of values intentionally chosen through a state of “reflective equilibrium”.
This extrapolation of our values, through Yudkowsky's Coherent Extrapolated Volition, was proposed as having some advantages when developing the first AI seed, and seems useful in thinking a set of machines ethics, namely:
::* the use of real human values after the reflective process;
::* faster AI moral progress; dissolving preference contradictions;
::* simplification of the human values through elimination of artifacts;
::* a possible solution for human goals’ integration in AI systems;
::* convergence of different human values.