I think this was a badly written post, and it appropriately got a lot of pushback.
Let my briefly try again: clarifying what I was trying to communicate.
Evolution did not succeed at aligning humans to the sole outer objective function of inclusive genetic fitness.
There are multiple possible reasons why evolution didn't succeed, and presumably multiple stacked problems.
But one thing that I've sometimes heard claimed or implied is that evolution couldn't possibly have succeeded at instilling inclusive genetic fitness as a goal, because individuals humans don't have inclusive genetic fitness as a concept.
Evolution could only have approximated that goal with a godshatter of adaptions to prefer various proxies to inclusive genetic fitness, where each proxy has to be close to the level of sensory-evidence. eg. Evolution can shape humans to like the taste of sugar, or the feeling of orgasm, or to prefer sexy-looking people, or even to love their cousins (less than their brothers but more than their more distant relatives). But, it's claimed, evolution can't shape humans to desire their own inclusive genetic fitness directly, because it can't instill goals that aren't at the at the level of sensory-evidence.
And so it's not surprising that the proxies would completely deviate from the "intended" target, as soon as conditions changed.
Before the 20th century, not a single human being had an explicit concept of "inclusive genetic fitness", the sole and absolute obsession of the blind idiot god. We have no instinctive revulsion of condoms or oral sex. Our brains, those supreme reproductive organs, don't perform a check for reproductive efficacy before granting us sexual pleasure.
Why not? Why aren't we consciously obsessed with inclusive genetic fitness? Why did the Evolution-of-Humans Fairy create brains that would invent condoms? "It would have been so easy," thinks the human, who can design new complex systems in an afternoon.
The Evolution Fairy, as we all know, is obsessed with inclusive genetic fitness. When she decides which genes to promote to universality, she doesn't seem to take into account anything except the number of copies a gene produces. (How strange!)
But since the maker of intelligence is thus obsessed, why not create intelligent agents - you can't call them humans - who would likewise care purely about inclusive genetic fitness? Such agents would have sex only as a means of reproduction, and wouldn't bother with sex that involved birth control. They could eat food out of an explicitly reasoned belief that food was necessary to reproduce, not because they liked the taste, and so they wouldn't eat candy if it became detrimental to survival or reproduction. Post-menopausal women would babysit grandchildren until they became sick enough to be a net drain on resources, and would then commit suicide.
Supposedly, evolution can't produce an inclusive genetic fitness maximizer, not just that it happened not to.
However, this story is undercut by an example in which evolution was able to make an abstract concept (not just a bunch of sensory correlates of that concept in the ancestral environment) an optimization target that the human will apply it's full creative intelligence to achieving.
Social status seems like one such an example.
It's an abstract concept that many humans have as an actual long term optimization target. They'll implement plans over years to increase their prestige, they don't just have a myopic status-grabbing heuristic. (One example: someone going to med school and residency for the prestigious role of being a doctor a decade later.)
And humans seem to have have a desire for social status itself, or at least not just for a collection of sensory-evidence-level proxy measures that correlated in the ancestral environment, and which break down entirely when the environment changes.
(If you doubt this, compare status-seeking behavior to male sexual preferences. In the latter case, it looks much more like evolution did instill a bunch of specific desires for close-to-sensory-level features that were proxies for fertility and health: big breasts, long legs, unwrinkled skin. Heterosexual men find those features desirable, and finding out that a particular sexy woman is actually infertile doesn't change the desirability.
But in the case of status-seeking, I can't write a list of collection of near-sensory-level features that that people desire, independently of actual social prestige. The markers of status are enormously varied, by culture and subculture, and constantly changing. I bet that Steve Byrnes can point out a bunch of specific sensory evidence that the brain uses to construct the status concept (stuff like gaze length of conspecifics or something?), but the human motivation system isn't just optimizing for those physical proxy measures, or people wouldn't be motivated to get prestige on internet forums where people have reputations but never see each other's faces.)
This is suggestive that at least in some circumstances, evolution actually can shape an organism to have at least a specific abstract concept as a long term optimization target, and recruit the organism's own intelligence to identifying how how that concept applies in many varied environments.
This is not to say that evolution succeeded at aligning humans. It didn't. This also doesn't imply that alignment is easy. Maybe it is, maybe it isn't, but this argument doesn't establish that.
But it is to say that the specific story for why evolution failed at aligning humans to inclusive genetic fitness that I believed in say 2020, is incorrect, or at least incomplete.
1
[Update: I think this post, as written, is confused. This comment is a hopefully clearer attempt at what I was aiming to express in this post.]
[This is post is a slightly edited tangent from my dialogue with John Wentworth here. I think the point is sufficiently interesting and important that I wanted to make it as a top level post, and not leave it buried in that dialog on mostly another topic.]
The conventional story is that natural selection failed extremely badly at aligning humans. One fact about humans that casts doubt on this story is that natural selection got the concept of "social status" into us, and it seems to have done a shockingly good job of aligning (many) humans to that concept.
Evolution somehow gave humans some kind of inductive bias (or something) such that our brains are reliably able to learn what it is to be "high status", even though the concrete markers for status are as varied as human cultures.
And it further, it successfully hooked up the motivation and planning systems to that "status" concept. Modern humans not only take actions that play for status in their local social environment, they sometimes successfully navigate (multi-decade) career trajectories and life paths, completely foreign to the ancestral environment, in order to become prestigious by the standards of the local culture.
And this is one of the major drivers of human behavior! As Robin Hanson argues, a huge portion of our activity is motivated by status-seeking and status-affiliation.
This is really impressive to me. It seems like natural selection didn't do so hot at aligning humans to inclusive genetic fitness. But it did kind of shockingly well aligning humans to the goal of seeking, even maximizing, status, all things considered.[1]
This seems like good news about alignment. The common story that condoms prove that evolution basically failed at alignment—that as soon as we developed the technological capability to route around the evolution's "goal" of maximizing the frequency of your alleles in the next generation, to attain only the proxy measure of sex, we did that—doesn't seem to apply to our status drive.
It looks to me like "status" generalized really well across the distributional shift of technological civilization. Humans still recognize it and optimize for it, regardless of whether the status markers are money or technical acumen or h-factor or military success.[2]
This makes me way less confident about the standard "evolution failed at alignment" story.
I guess that we can infer from this that having an intuitive "status" concept was much more strongly instrumental for attaining high inclusive genetic fitness in the ancestral environment than having an intuitive concept of "inclusive genetic fitness" itself. A human-level status seeking agent with a sex drive does better by the standard of inclusive genetic fitness than a human-level agent IGF maximizer.
The other hypothesis, of course, is that the "status" concept was easier to encode in an human than the "inclusive genetic fitness concept, for some reason.
I'm interested if others think that this is an illusion, if it only looks like the status target generalized, because I'm drawing the target around where the arrow landed. That is, what we think of as "social status" is exactly the parts of social status in the ancestral environment that did generalize to across cultures.