Content warning: taboo topics
From "complex adaptations require a sequence of simpler adaptations", we cannot conclude "that a single human cannot have a complex adaptation that is not universal"! What you are missing is physical barriers that existed for large parts of human history and prehistory, such as the Sahara Desert, that largely prevented any significant amount of gene flow. While individual early European explorers could cross the Sahara Desert and might even have survived for a short while, they would face considerable dangers on the other side: tropical diseases such as malaria to which their bodies and immune systems in particular were poorly adapted, no modern science-based medicine to treat them with and no easy way to communicate with locals, unfamiliar wild animals such as the very dangerous hippopotamus, and hostile African tribes.
And yes, African tribes would probably have been highly suspicious of, and hostile towards, white outsiders - not least because of their "strange-coloured" skin, strange clothes, and incomprehensible language. Even today, Albino Africans are discriminated against in Africa, and sometimes even killed for their flesh to be used by witch-doctors. The modern notion that we should care about all humans equally, because racial and national differences are superficial at most, is quite a modern phenomenon. There is little sign of these kinds of ideas in most of human history, although Islam approaches this idea with the idea of the Ummah (brotherhood of all Muslim men and sisterhood of all Muslim women) and Christianity gets even closer with the idea of people of other races (the Good Samaritan, for example) being "your neighbour" who should be treated "as yourself". Both were quite radical breaks from the more tribal pasts that had preceded them - and we can see that in evidence in the Old Testament. The God of the Old Testament is portrayed as snuffing out entire rival tribes - or in one case, the Great Flood story, the entire outside world. But of course, the period I am referring to was before European imperialism spread Christianity in Africa. (And in practice, Christian ideals of treating others as neighbours were not in evidence in the Crusades, or in the many expulsions of Jews from European countries.)
One cause of variation in intelligence is injuries, such as head injuries. (It is a little-known fact that this can also cause personality changes, such as increased risk-taking - so there is an intriguing hypothesis that the prevalence of both diagnosed and undiagnosed blunt-force head trauma caused by male students playing American Football may have led to some men randomly obtaining the "gift" of risk-taking, leading to greater US entrepreneurship, and thus contributed to America's impressive economic performance. My confidence in this hypothesis is very low, however, because I have only seen it mentioned on Twitter.)
However, it is noteworthy that while injuries, such as brain injuries caused by temporary loss of oxygen during a badly-managed birth, or caused by blunt-force trauma, can cause a reduction in IQ, the "moron level" or "idiot level" - the IQ level at which someone is considered developmentally disabled - can differ by population group. It is an unpalatable fact that a white European with an IQ of 60 and the typical impairments going along with that would probably be considered developmentally disabled, while a pygmy hunter-gatherer with an IQ of 60 - normalised to the same scale as the European - might not. Although the pygmy, in line with other traditional hunter-gatherer peoples that survive today, perhaps does not read or write or count above ten in their native language, and spends a lot of time hunting, gathering, or trying to sell what they have hunted, and thus the cognitive demands upon them in their daily lives would arguably be a bit lower, this difference in assessments is mainly because the IQ averages of the two groups are significantly different. Why might we expect this to be so? Well, as already noted, different population groups were semi-completely separated by physical barriers, such as the Sahara Desert and the ocean around Australia, for long periods of time. They also faced different selection pressures - another factor making substantial group differences plausible from an evolutionary perspective. Recall that the majority of experts believe that homo sapiens originated in Africa, so if that's true, all races have ancestors there - so white and Asian people must have had ancestors who lived in Africa, some of whom must have migrated out of Africa at some point. (And by the way, we have research on how each population group on earth got where they reside today, including Native Americans and Australian Aborigines.) The development of farming - obviously a technological factor, but one that was sort of necessitated by migrating into harsher environments with harsher winters - may have exacerbated these selection pressures.
Hunter-gathering requires a certain amount of intelligence, clearly, but perhaps less so than farming. Farming, particularly in challenging environments with long, heavy winters with snow on the ground, requires more long-term thinking and values, and rewards intelligence that produces innovations that yield more food more reliably - in the Malthusian era that comprised most of human history and prehistory, those who were able to have and feed and protect more children because of their superior food-obtaining abilities, would often have passed on more copies of their genes, and those that had less success at doing so would have passed on fewer copies of their genes, or even none at all. This is so even though such technological innovations will have spread out over time, thanks to the one intelligence characteristic that dramatically differs between humans and all other species - the ability to learn from and teach each other. (Note that many humans throughout Malthusian times never had any surviving children - whether due to infertility, extreme poverty or injuries acquired from war or other violence.)
But the group differences don't stop there. Certain specific population groups, such as Ashkenazi Jews in Europe, the Igbo tribes in Nigeria, and - as recent indirect findings by Gregory Clark have indicated - in modern times, the Southern English (which includes Northern English individuals who migrated, or whose ancestors migrated, to the South of England), seem to have higher average intelligence than the average of the broader groups they live next to, despite not having been physically separated from other groups in quite the same way. On the other end of the intelligence spectrum, though this is doubtless somewhat embarrassing for at least some people with such ancestry, there seems to be correlations in humans between brain size and IQ, between height and IQ, and between height and brain size, and the observed intelligence of certain population groups with shorter average height seems to bear this correlation out. (However, it is important to note that women are not substantially less intelligent than men on average, despite being significantly shorter on average. I would speculate that perhaps women have a similar total number of neurons to men, but packed in more densely on average - in line with how certain other parts of their bodies are often genetically programmed to be smaller than, larger than or structurally different to men's. Animal research has established that an underlying factor influencing intelligence on an inter-species level is number of neurons, rather than brain size per se - which explains why whales and elephants are not more intelligent than humans.)
Why might this be so? For Ashkenazi Jews - that is, Jews of proximate European ancestry - we don't know for sure. However, one plausible hypothesis holds that that the regulatory environment over the course of hundreds of years of living in European nations, where Jewish people were frequently the only people allowed to levy interest-bearing loans due to the then-in-force Christian prohibition on usury, together with high-pressure sexual selection and notably arranged marriages, would have selected for successful and intelligent people and their genes.
Note that this probably seems very implausible from the perspective of the idea, common among non-scientists in the West, that evolution takes a very, very long time to do anything, especially to produce complex traits, a conception of evolution which has probably been influenced by the highly influential essaying of the anti-hereditarian (i.e. opposed to the idea that there are genetically-caused differences in average IQ between groups) Steven Jay Gould. But actually we have an existence proof that selection doesn't have to take a long time to produce changes from the human development of dog breeds using artificial selection, and of cows bred to have unnaturally large udders to maximise milk production and so on, and scientific research on species with much shorter generation lengths than our own bears this out as well. And even Gould himself was a proponent of the theory of punctuated equilibrium, which literally means slow periods where nothing much happened punctuated by periods of comparatively rapid change. And if you think about it, there are an incredibly large number of adaptations required to turn simple microorganisms into modern humans, both ones currently known to science and presumably a large number not known to science yet, because of our comparative lack of understanding of the human brain, so it's not the case that each and every one of those changes would have taken millions of years - there simply wouldn't have been enough time.
In addition, being able to memorise and interpret large portions of the scriptures was highly valued, which would plausibly have selected for verbal abilities, which is anecdotally extraordinarily high among Ashkenazi Jews (I know an Ashkenazi Jewish man who speaks several languages fluently - including English, which is not his first language, with flawless pronunciation). Indeed, not only would more successful Jewish financiers, religious scholars and rabbis on average have had more children - less successful Jewish men, if they had reproduced at all, might have simply out-married from the group and therefore their heirs would not have been considered Jewish. This would be despite having the same proportion of Jewish blood - 50% - as a child of a Jewish female and a Christian convert, who would be considered Jewish under traditional Jewish law, and would of course be raised as part of the Jewish community. So the whole traditional demarcation between Jewish and non-Jewish people, defined by "mother is Jewish", technically makes little sense from a modern scientific perspective with the benefit of our knowledge of how genetics works, but it could help to form part of the story of how the average intelligence of Jewish people living in Europe apparently rose more than their non-Jewish peers despite not-inconsiderable intermarriage in both directions over the centuries (which is probably why Jewish and non-Jewish people of European ancestry are often difficult to distinguish visually, in the absence of clothing or grooming cues in the case of Orthodox Jewish men).
Less is known about the history of the Igbo tribes, who were historically farmers, and even less about the past of individual hunter-gatherer tribes.
In England, the South is more prosperous due to the very economically-important city of London, in modern times one of the world's top financial centres, but which has also been a key global city for centuries. So there would have been strong economic incentives to migrate for work, or a better or more reliable income, over the course of a number of centuries - especially in centuries past when the equivalents of "welfare" (in British English, "benefits"), were slim or inadequate compared to today, or when there was no year-round work available in the local village, with its relatively simple economy. I don't know how open to migration the English are on average, but Americans are unusually prone to migrate for a better job (another possible reason for unusually high salaries in America, I would speculate), and many Americans have English ancestry, of course. So perhaps selection effects alone could explain the apparent average IQ differences within England.
Yishan Wong, the former CEO of Reddit, already has, so you might ask him - but I suggest you read all his blog posts on the subject first so that you don't ask anything that he's already covered.