Very interesting post! However, I have a big disagreement with your interpretation of why the European conquerors succeeded in America, and I think that it undermines much of your conclusion.
In your section titled "What explains these devastating takeovers?" you cite technology and strategic ability, but Old World diseases destroyed the communities in America before the European invaders arrived, most notably smallpox, but also measles, influenza, typhus and the bubonic plague. My reading of historians (from Charles Mann's book 1493, to Alfred W. Crosby's The Columbian Exchange and Jared Diamond's Guns Germs and Steel) leads me to conclude that the historical consensus is that the main reason for all of these takeovers was due to Old World diseases, and had relatively little to do with technology or strategy per se.
In Chapter 11 of Guns Germs and Steel, Jared Diamond analyzes the European takeovers in America you cite from the perspective of old World diseases (Here's a video from a Youtuber named CGP Grey who made a video on the same topic). The basic thesis is that Europeans had acquired immunity from these diseases, whereas people in America hadn't. From Wikipedia,
After first co...
This is a good critique; thank you.
I have two responses, and then a few nitpicks.
First response: Disease wasn't a part of Afonso's success. It helped the Europeans take over the Americas but did not help them take over Africa or Asia or the middle east; this suggests to me that it may have been a contributing factor but was not the primary explanation / was not strictly necessary.
Second response: Even if we decide that Cortes and Pizarro wouldn't have been able to succeed without the disease, my overall conclusion still stands. This is because disease isn't directly the cause of conquistador success, but rather indirectly, via the intermediate steps of "Chaos and disruption" and "Reduced political and economic strength." (I claim.) And the reduced strength can't have been more than, say, a 90% reduction in strength. (I claim) Suppose we think of the original conclusion as "A force with a small tech and cunning/experience advantage can take over a region 10,000 times its size." Then the modified conclusion in light of your claim about disease would be "In times of chaos and disruption, a force with a small tech and cunning...
Is this a fair description of your disagreement re the 90% argument?
Daniel thinks that a 90% reduction in the population of a civilization corresponds to a ~90% reduction in their power/influentialness. Because the Americans so greatly outnumbered the Spanish, this ten-fold reduction in power/influentialness doesn’t much alter the conclusion.
Matthew thinks that a 90% reduction in the population of a civilization means that “you don’t really have a civilization”, which I interpret to mean something like a ~99.9%+ reduction in the power/influentialness of a civilization, which occurs mainly through a reduction in their ability to coordinate (e.g. “chain of command in ruins”). This is significant enough to undermine the main conclusion.
If this is accurate, would a historical survey of the power/influentialness of civilisations after they lose 90% of the population (inasmuch as these cases exist) resolve the disagreement?
Another example in this vein is Robert Clive's takeover of the Bengal Sultanate. As Nick Robins documents in The Corporation that Changed the World, Robert Clive was sent to Bengal by the British East India Company with the instruction to set up a modest trading outpost and specifically not engage in local politics or intrigue. So, of course, he intrigues with Mir Jafar, the commander in chief of the Nawab's army and gets him to change sides (along with a large portion of the Nawab's army) during the Battle of Plassey.
The East India Company's victory at the battle of Plassey resulted in immediate financial gain of £2,500,000 (now valued at over £250,000,000) for the Company, plus another £234,000 (now valued at £23,000,000) for Clive himself. Even more importantly, the EIC gained the right of diwani -- the right to collect taxes on behalf of the Mughals in Bengal. This allowed them to establish themselves as a commercial monopoly over the Bengal textile industry (considered the best in the world at that time) and further entrench themselves as a commercial and military power in India. For good reason, the Battle of Plassey is seen as a key moment in the consolidation of the British Empire in India, even though it was fought by a private individual on behalf of a private corporation.
Some more thoughts that came to me after I posted this:
1. I should have mentioned who I take myself to be responding to. I'm responding to claims I've heard made by various people (Katja Grace and Paul Christiano, to name two very smart examples) that I summarize as follows: "For one entity to take over the world, it needs more power than the rest of the world combined. That means it must either start out with a lot of power -- e.g. a major nation-state like China or USA -- or grow in power much faster than the rest of the world for a while. The latter possibility could happen if an intelligence explosion occurred in one local area, but this is unlikely; the intelligence explosion is more likely to be distributed fairly broadly."
2. I think these case studies are also historical precedents for "Treacherous Turns." Cortes and Pizarro were certainly very treacherous. (I don't think Afonso was, but I'd have to reread the stories.)
3. I have vague, conspiracy-theory-esque worries that actually the conquistadors really did just all get lucky, and that the explanation for this is some sort of anthropic selection effect. I don't think this is pl...
(I am the author)
I still like & endorse this post. When I wrote it, I hadn't read more than the wiki articles on the subject. But then afterwards I went and read 3 books (written by historians) about it, and I think the original post held up very well to all this new info. In particular, the main critique the post got -- that disease was more important than I made it sound, in a way that undermined my conclusion -- seems to have been pretty wrong. (See e.g. this comment thread, these follow up posts)
So, why does it matter? What contribution did this post make? Well, at the time -- and still now, though I think I've made a dent in the discourse -- quite a lot of people I respect (such as people at OpenPhil) seemed to think unaligned AGI would need god-like powers to be able to take over the world -- it would need to be stronger than the rest of the world combined! I think this is based on a flawed model of how takeover/conquest works, and history contains plenty of counterexamples to the model. The conquistadors are my favorite counterexample from my limited knowledge of history. (The flawed model goes by the name of "The China Argument," at least in my mind. You may have heard ...
I think technological advantage - specifically sailing technology - probably played a much larger role in Afonso's takeover than it would seem from a quick read. Key pieces:
Monsoons: wind around India blows consistently Southwest for half the year, and Northeast for the other half. IIRC from Braudel, this made trade in the Indian ocean highly predictable: everyone sailed with the wind at their back and ran consistent one-year circuits. As you mention:
The Indian Ocean contained most of the world's trade at the time, since it linked up the world's biggest and wealthiest regions.
I'd guess that the monsoons were probably a bigger factor here than vicinity to wealthy regions. In particular:
Europe is just coming out of the Middle Ages and does not have an obvious technological advantage over India or China or the Middle East, and has an obvious economic disadvantage.
Europe had less total wealth (because it had a smaller population) and was behind technologically in some ways (e.g. metallurgy), but even in the 15th-16th century Europe was considered "wealthy" on a per-capita basis. In particular, Europe had much more per-capita capital good...
Nice post. Were there any sources besides Wikipedia that you found especially helpful when researching this post?
However, I don't think this is the whole explanation. The technological advantage of the conquistadors was not overwhelming.
With regard to the Americas at least, I just happened to read this article by a professional military historian, who characterizes the Native American military technology as being "thousands of years behind their Old World agrarian counterparts", which sounds like the advantage was actually rather overwhelming.
...There is a massive amount of literature to explain what is sometimes called ‘the Great Divergence‘ (a term I am going to use h
I just listened to a podcast on the fall of the Inca. Two points stood out that seem relevant.
One is that the Inca themselves conquered the area through a combination of force and diplomacy. Not, I think, as thoroughly? Not sure how I'd operationalize that, and I'm not entirely sure why I think it deserves emphasis. Perhaps... this makes it clearer to me that in one more sense, the Spanish weren't "playing on easy mode"; the Inca rose to the top of the local power game, and then the Spanish beat them.
Another is that the Incan emperor wasn't entirely helple...
Based on what I recall reading about Pizzaro's conquest, I feel you might be underestimating the importance of horses. It took centuries for European powers to figure out how to break a heavy cavalry charge with infantry; the amerindians didn't have the time to figure it out (see various battles where small cavalry forces routed thousands of troops). Once they had got more used to horses, later Inca forces (though much diminished) were more able to win open battles against the Spanish.
Maybe this was the problem for these empires: they were used to winning
...Here's what I'll be putting in the Alignment Newsletter about this piece. Let me know if you spot inaccuracies or lingering disagreement regarding the opinion section.
Summary:
This post lists three historical examples of how small human groups conquered large parts of the world, and shows how they are arguably precedents for AI takeover scenarios. The first two historical examples are the conquests of American civilizations by Hernán Cortés and Francisco Pizarro in the early 16th century. The third example is the Portugese capture of key...
Promoted to curated: I really like this post. I already linked to it two times, and it clearly grounds some examples that I've seen people use informally in AI Alignment discussions in a way that will hopefully create a common reference, and allow us to analyze this kind of argument in much more detail.
There is a sci-fi sourcebook called 'GURPS Ogre' about an AI dominated future that follows a similar line of reasoning, I think OP might enjoy it, and pdfs can be found online.
I also think that the story of Napoleon's conquest and the reasons for its' success might also be informative to your thesis, as disease is not (at least as far as I know) nearly as much of a factor.
Another thought:
Japan was in a similar predicament in the mid-1850s, but successfully managed not only to survive but to thrive, modernizing their economy and military to Western levels without suffering regime change in the process.
What explains their success? It's not that the technological gap between them and Western powers was smaller than the technological gap between the conquistadors and their victims. (I think.) Rather, it's that there was no "diplomatic and strategic cunning" gap, and that was because there was no experience g...
How significant do you think it was that the conquistadors had access to a deep historical archive? The Aztecs and Incas arguably had writing, but it wasn't used to preserve a large library of historical and fictional stories. The Portuguese presumably were reasonably educated, and knew many stories of emperors and empires. The South Americans may have had some oral histories to go by, but I presume it was paltry in comparison.
The Indian case was far different, I would imagine, so this hypothesis doesn't touch that case at all.
NEW EDIT: After reading three giant history books on the subject, I take back my previous edit. My original claims were correct.
Could you edit this comment to add which three books you're referring to?
In 1492, Iberian Christians had finally defeated the last remnant of the Moorish Empire that had subjugated most of their peninsula. The Reconquista. It was an enormous achievement against a powerful and advanced foe. [It also occasioned the expulsion of the Sephardic Jews, an act both odious and self-wounding.] Anyway, in the process of defeating the Moors, the Iberian Christian armed forces had become really really good at fighting. Brave unto reckless, for one thing. For another, they were able to count on a deep bench of sub-commanders and captains who...
Related reading - https://mattlakeman.org/2020/06/25/polygamy-human-sacrifices-and-steel-why-the-aztecs-were-awesome/
The "experience" and literacy ideas can draw a straight line to Hardcore History podcaster Dan Carlin's ideas on "intellectual contagion." In particular, how the working classes that comprised all armies in WWI began to throw off the idea that the aims of their monarchs or empires superseded their own well-being ... and how that affected the latter stages of the war.
Going further, 9/11 immediately ushered in an era of TSA, reinforced cockpit doors and air marshals that, one can argue, were already superfluous. Before the day's...
Crossposted from AI Impacts.
Epistemic status: I am not a historian, nor have I investigated these case studies in detail. I admit I am still uncertain about how the conquistadors were able to colonize so much of the world so quickly. I think my ignorance is excusable because this is just a blog post; I welcome corrections from people who know more. If it generates sufficient interest I might do a deeper investigation. Even if I’m right, this is just one set of historical case-studies; it doesn’t prove anything about AI, even if it is suggestive. Finally, in describing these conquistadors as “successful,” I simply mean that they achieved their goals, not that what they achieved was good.
Summary
In the span of a few years, some minor European explorers (later known as the conquistadors) encountered, conquered, and enslaved several huge regions of the world. That they were able to do this is surprising; their technological advantage was not huge. (This was before the scientific and industrial revolutions.) From these cases, I think we learn that it is occasionally possible for a small force to quickly conquer large parts of the world, despite:
Which all suggests that it isn’t as implausible that a small AI takes over the world in mildly favorable circumstances as is sometimes thought.
EDIT: In light of good pushback from people (e.g. Lucy.ea8 and e.g. Matthew Barnett) about the importance of disease, I think one should probably add a caveat to the above: "In times of chaos & disruption, at least."
NEW EDIT: After reading three giant history books on the subject, I take back my previous edit. My original claims were correct.
Three shocking true stories
I highly recommend you read the wiki pages yourself; otherwise, here are my summaries:
Cortés: [wiki] [wiki]
Pizarro [wiki] [wiki]
Afonso [wiki] [wiki] [wiki]
Some comparisons and contrasts:
What explains these devastating conquests?
Wrong answer: I cherry-picked my case studies.
History is full of incredibly successful conquerors: Alexander the Great, Ghenghis Khan, etc. Perhaps some people are just really good at it, or really lucky, or both.
However: Three incredibly successful conquerors from the same tiny region and time period, conquering three separate empires? Followed up by dozens of less successful but still very successful conquerors from the same region and time period? Surely this is not a coincidence. Moreover, it’s not like the conquistadors had many failed attempts and a few successes. The Aztec and Inca empires were the two biggest empires in the Americas, and there weren’t any other Indian Oceans for the Portuguese to fail at conquering.
Fun fact: I had not heard of Afonso before I started writing this post this morning. Following the Rule of Three, I needed a third example and I predicted on the basis of Cortes and Pizarro that there would be other, similar stories happening in the world at around that time. That’s how I found Afonso.
Right answer: Technology
However, I don't think this is the whole explanation. The technological advantage of the conquistadors was not overwhelming.
Whatever technological advantage the conquistadors had over the existing empires, it was the sort of technological advantage that one could acquire before the Scientific and Industrial revolutions. Technology didn't change very fast back then, yet Portugal managed to get a lead over the Ottomans, Egyptians, Mughals, etc. that was sufficient to bring them victory. On paper, the Aztecs and Spanish were pretty similar: Both were medieval, feudal civilizations. I don't know for sure, but I'd bet there were at least a few techniques and technologies the Aztecs had that the Spanish didn't. And of course the technological similarities between the Portuguese and their enemies were much stronger; the Ottomans even had access to European mercenaries! Even in cases in which the conquistadors had technology that was completely novel -- like steel armor, horses, and gunpowder were to the Aztecs and Incas -- it wasn't god-like. The armored soldiers were still killable; the gunpowder was more effective than arrows but limited in supply, etc.
(Contrary to popular legend, neither Cortés nor Pizarro were regarded as gods by the people they conquered. The Incas concluded pretty early on that the Spanish were mere men, and while the idea did float around the Aztecs for a bit the modern historical consensus is that most of them didn't take it seriously.)
Ask yourself: Suppose Cortés had found 500 local warriors, gave them all his equipment, trained them to use it expertly, and left. Would those local men have taken over all of Mexico? I doubt it. And this is despite the fact that they would have had much better local knowledge than Cortés did! Same goes for Pizarro and Afonso. Perhaps if he had found 500 local warriors led by an exceptional commander it would work. But the explanation for the conquistador’s success can’t just be that they were all exceptional commanders; that would be positing too much innate talent to occur in one small region of the globe at one time.
Right answer: Strategic and diplomatic cunning
This is my non-expert guess about the missing factor that joins with technology to explain this pattern of conquistador success.
They didn't just have technology; they had effective strategy and they had effective diplomacy. They made long-term plans that worked despite being breathtakingly ambitious. (And their short-term plans were usually pretty effective too, read the stories in detail to see this.) Despite not knowing the local culture or history, these conquistadors made surprisingly savvy diplomatic decisions. They knew when they could get away with breaking their word and when they couldn't; they knew which outrages the locals would tolerate and which they wouldn’t; they knew how to convince locals to ally with them; they knew how to use words to escape militarily impossible situations… The locals, by contrast, often badly misjudged the conquistadors, e.g. not thinking Pizarro had the will (or the ability?) to kidnap the emperor, and thinking the emperor would be safe as long as they played along.
This raises the question, how did they get that advantage? My answer: they had experience with this sort of thing, whereas locals didn't. Presumably Pizarro learned from Cortés' experience; his strategy was pretty similar. (See also: the prior conquest of the Canary Islands by the Spanish). In Afonso's case, well, the Portuguese had been sailing around Africa, conquering ports and building forts for more than a hundred years.
Lessons I think we learn
I think we learn that:
It is occasionally possible for a small force to quickly conquer large parts of the world, despite:
Which all suggests that it isn’t as implausible that a small AI takes over the world in mildly favorable circumstances as is sometimes thought.
EDIT: In light of good pushback from people (e.g. Lucy.ea8 and e.g. Matthew Barnett) about the importance of disease, I think one should probably add a caveat to the above: "In times of chaos & disruption, at least."
Having only a minuscule fraction of the world's resources and power
In all three examples, the conquest was more or less completed without support from home; while Spain/Portugal did send reinforcements, it wasn't even close to the entire nation of Spain/Portugal fighting the war. So these conquests are examples of non-state entities conquering states, so to speak. (That said, their claim to represent a large state may have been crucial for Cortes and Pizarro getting audiences and respect initially.) Cortés landed with about a thousandth the troops of Tenochtitlan, which controlled a still larger empire of vassal states. Of course, his troops were better equipped, but on the other hand they were also cut off from resupply, whereas the Aztecs were in their home territory, able to draw on a large civilian population for new recruits and resupply.
The conquests succeeded in large part due to diplomacy. This has implications for AI takeover scenarios; rather than imagining a conflict of humans vs. robots, we could imagine humans vs. humans-with-AI-advisers, with the latter faction winning and somehow by the end of the conflict the AI advisers have managed to become de facto rulers, using the humans who obey them to put down rebellions by the humans who don't.
Having technology + diplomatic and strategic skill that is better but not that much better
As previously mentioned, the conquistadors didn’t enjoy god-like technological superiority. In the case of Afonso the technology was pretty similar. Technology played an important role in their success, but it wasn’t enough on its own. Meanwhile, the conquistadors may have had more diplomatic and strategic cunning (or experience) than the enemies they conquered. But not that much more--they are only human, after all. And their enemies were pretty smart.
In the AI context, we don't need to imagine god-like technology (e.g. swarms of self-replicating nanobots) to get an AI takeover. It might even be possible without any new physical technologies at all! Just superior software, e.g. piloting software for military drones, targeting software for anti-missile defenses, cyberwarfare capabilities, data analysis for military intelligence, and of course excellent propaganda and persuasion.
Nor do we need to imagine an AI so savvy and persuasive that it can persuade anyone of anything. We just need to imagine it about as cunning and experienced relative to its enemies as Cortés, Pizarro, and Afonso were relative to theirs. (Presumably no AI would be experienced with world takeover, but perhaps an intelligence advantage would give it the same benefits as an experience advantage.) And if I’m wrong about this explanation for the conquistador’s success--if they had no such advantage in cunning/experience--then the conclusion is even stronger.
Additionally, in a rapidly-changing world that is undergoing slow takeoff, where there are lesser AIs and AI-created technologies all over the place, most of which are successfully controlled by humans, AI takeover might still happen if one AI is better, but not that much better, than the others.
Having very little data about the world when the conquest begins
Cortés invaded Mexico knowing very little about it. After all, the Spanish had only realized the Americas existed two decades prior. He heard rumors of a big wealthy empire and he set out to conquer it, knowing little of the technology and tactics he would face. Two years later, he ruled the place.
Pizarro and Afonzo were in better epistemic positions, but still, they had to learn a lot of important details (like what the local power centers, norms, and conflicts were, and exactly what technology the locals had) on the fly. But they were good at learning these things and making it up as they went along, apparently.
We can expect superhuman AI to be good at learning. Even if it starts off knowing very little about the world -- say, it figured out it was in a training environment and hacked its way out, having inferred a few general facts about its creators but not much else -- if it is good at learning and reasoning, it might still be pretty dangerous.
Being disunited
Cortés invaded Mexico in defiance of his superiors and had to defeat the army they sent to arrest him. Pizarro ended up fighting a civil war against his fellow conquistadors in the middle of his conquest of Peru. Afonzo fought Greek mercenaries and some traitor Portuguese, conquered Malacca against the orders of a rival conquistador in the area, and was ultimately demoted due to political maneuvers by rivals back home.
This astonishes me. Somehow these conquests were completed by people who were at the same time busy infighting and backstabbing each other!
Why was it that the conquistadors were able to split the locals into factions, ally with some to defeat the others, and end up on top? Why didn't it happen the other way around: some ambitious local ruler talks to the conquistadors, exploits their internal divisions, allies with some to defeat the others, and ends up on top?
I think the answer is partly the "diplomatic and strategic cunning” mentioned earlier, but mostly other things. (The conquistadors were disunited, but presumably were united in the ways that mattered.) At any rate, I expect AIs to be pretty good at coordinating too; they should be able to conquer the world just fine even while competing fiercely with each other. For more on this idea, see this comment.
By Daniel Kokotajlo
Acknowledgements
Thanks to Katja Grace for feedback on a draft. All mistakes are my own, and should be pointed out in the comments. Edit: Also, when I wrote this post I had forgotten that the basic idea for it probably came from this comment by JoshuaFox.