He looked at my diagram showing the "village idiot" next to "Einstein", and said, "That seems wrong to me; I think Einstein should be way off on the right."
We can distinguish between system I and system II abilities (http://web.cenet.org.cn/upfile/37554.pdf). Einstein and the village idiot share most of their system I abilities. For example: They learned the complex syntax and semantics of their respective native languages effortlessly as children without needed explicit tuition. They both mastered basic human folk psychology / theory of mind including reasoning about desire and belief ascriptions and motivation. They both are competent with standard human folk physics (involving recognition of objects as discrete, crude mechanics, etc.). They both have a basic competence in terms of picking up their native culture (e.g. etiquette, moralistic/religious taboos, hierarchy, simple arts and religion).
Now, non-human animals possess some of these system I abilities. However, a fair amount of the human language, folk psychology and culture abilities may be well beyond those of chimps, bonobos, etc.
Einstein and the village idiot may differ more significantly in system II abilities, i.e. conscious reasoning. My experience of people good at conscious reasoning in multiple domains is that they can do more good conscious reasoning (both performing analysis and synthesis)in 30 minutes than an average IQ person (NOT a village idiot) has in a lifetime. Thus, in terms of system II abilities, it might be that Einstein is further from the village idiot (relative to the distance between the idiot and the chimp) than Eliezer's diagram suggests.
Evolutionary Psychology stresses the uniformity of human cognitive abilities, suggesting something like Eliezer's diagram. But I'm skeptical that this uniformity extends to system II. The system II abilities of the best rationalists of today may depend significantly on their having learned a set of reasoning skills developed by their culture over a long period of time. The learning of these skills requires more basic abilities (g factor, etc.) but once these skills have been mastered the resulting difference in system II analytical and creative reasoning is much larger than the difference in Spearman's g. Another reason for an objectively huge range of human abilities in system II comes from human general learning capacities (which may significantly exceed those of our primate relatives). Top rationalists can spend hours a day (every day) engaged in focused system II reasoning. They probably do as much in a day as the idiot does in six months.
"But I'm skeptical that this uniformity extends to system II. The system II abilities of the best rationalists of today may depend significantly on their having learned a set of reasoning skills developed by their culture over a long period of time."
That's precisely the point; the biological difference between humans is not that great, so the huge differences we see in human accomplishment must be due in large part to other factors.
Agreed. But I think that if you put up a scale of "intelligence" then people will take into account abilities other than those that are (in some ill-defined and crude sense) dependent only on biology. And if we're talking about building an AI, then I'm not sure how useful it is to attempt to distinguish biological and other factors. If Feynman or Von Neumann or Pauling really are a bigger distance from the VI than Eliezer allows in terms of their system II thinking ability, then that seems to me significant independently of whether that distance is best explained in terms of the interaction of powerful learned thinking tools (scientific method, analogical reasoning, logical argument) with high g factor rather than the g factor on its own.