Is Friendly AI or more specifically CEV predicated on Eliminative Materialism being false? To what extent is FAI predicated on folk psychological theories of mental content turning out to accurately reflect human neurobiology?
From the article:
Modern versions of eliminative materialism claim that our common-sense understanding of psychological states and processes is deeply mistaken and that some or all of our ordinary notions of mental states will have no home, at any level of analysis, in a sophisticated and accurate account of the mind. In other words, it is the view that certain common-sense mental states, such as beliefs and desires, do not exist.
CEV is based on the idea that there is an algorithm that can look at the state of my brain, filter out various kinds of noise, and extrapolate what sort of desires and values I'd want to have if I lived in a kinder more benevolent society, wasn't subject to nearly as many serious cognitive biases, etc.
The problem I'm seeing is that the origin and meaning of terms like 'desire' and 'value' are in prescientific culture - folk psychology. they were created by people in absolute ignorance about how brains work,... (read more)