TekhneMakre

There is an argument that although humans evolved under pressure to maximize inclusive genetic fitness (IGF), humans don't actually try to maximize their own IGF. This, as the argument goes, shows that in the one case we have of a process creating general intelligence, it was not the case that the optimization target of the created intelligence ended up being the same as the optimization target of the process that created it. Therefore, alignment doesn't happen by default. To quote from A central AI alignment problem: capabilities generalization, and the sharp left turn: > And in the same stroke that [an AI's] capabilities leap forward, its alignment properties are revealed to be shallow, and to fail to generalize. The central analogy here is that optimizing apes for inclusive genetic fitness (IGF) doesn't make the resulting humans optimize mentally for IGF. Like, sure, the apes are eating because they have a hunger instinct and having sex because it feels good—but it's not like they could be eating/fornicating due to explicit reasoning about how those activities lead to more IGF. They can't yet perform the sort of abstract reasoning that would correctly justify those actions in terms of IGF. And then, when they start to generalize well in the way of humans, they predictably don't suddenly start eating/fornicating because of abstract reasoning about IGF, even though they now could. Instead, they invent condoms, and fight you if you try to remove their enjoyment of good food (telling them to just calculate IGF manually). The alignment properties you lauded before the capabilities started to generalize, predictably fail to generalize with the capabilities. Jacob published Evolution Solved Alignment (what sharp left turn?), arguing that actually humans represent a great alignment success. Evolution was trying to make things that make many copies of themselves, and humans are enormously successful on that metric. To quote Jacob: > For the evolution of human intelli