Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

Concept extrapolation is the skill of taking a concept, a feature, or a goal that is defined in a narrow training situation... and extrapolating it safely to a more general situation. This more general situation might be very extreme, and the original concept might not make much sense (eg defining "human beings" in terms of quantum fields).

Nevertheless, since training data is always insufficient, key concepts must be extrapolated. And doing so successfully is a skill that humans have to a certain degree, and that an aligned AI would need to possess to a higher extent.

This sequence collects the key posts on concept extrapolation. They are not necessarily to be read in this order; different people will find different posts useful.

New Comment
2 comments, sorted by Click to highlight new comments since: Today at 4:10 PM

I have a comment here that argues many patterns in human values and our generalizations of values emerge from an inner alignment failure in the brain. I’d be interested in hearing your perspective on it and whether it tracks with your own thinking on concept extrapolation.

Thanks for that link. It does seem to correspond intuitively to a lot of the human condition. Though it doesn't really explain value extrapolation, more the starting point from which humans can extrapolate values. Still a fascinating read, thanks!

New to LessWrong?