Lessons on AI Takeover from the conquistadors

by Daniel Kokotajlo, jacobjacob5 min read17th Jul 202030 comments

53

HistoryLessWrong Event TranscriptsAI
Frontpage

(Talk given at an event on Sunday 28th of June. Daniel Kokotajlo is responsible for the talk, Jacob Lagerros and David Lambert edited the transcript. 

If you're a curated author and interested in giving a 5-min talk, which will then be transcribed and edited, sign up here.) 

 

Daniel Kokotajlo: I'm basically going to recap my conquistadors posts. I'm interested to hear what people think about it. I know a lot more about the situation now because I've been reading some history books. But my overall opinion hasn't changed. 

I wrote this post because in talking to various people and reading some things, there was what seemed to me to be a simplistic model of how military conflicts work, or how a military takeover would work. Here's the simple model.

If  we're imagining a scenario where an AI appears, it will likely be in some lab somewhere. So it will start off with very few resources, very small amounts of money, for example. So you would have to have some sort of god-like intelligence or a technological advantage in order to have more strength than the rest of the world combined.

I think the conquistadors are counterexamples to this. So for a brief overview, I picked out three conquistadors. I had already known about Cortes and Pizarro, which were probably the most extreme examples. But I predicted that there would be a general trend here, so I did some Googling and after 20 minutes I found Alfonso, who is another example of this trend.

 

Alfonso was a Portuguese explorer general. Here is a map of his exploration. 

Ben Pace: Can I ask what period of history this was?

Daniel Kokotajlo: This is all around the early 1500s. Alfonso was 1506 to 1513, and then Cortes was 1519 and Pizarro was 1530.

If you look at this map, basically, Columbus went across the Atlantic in 1492, and the Portuguese were, in the same time, exploring down the coast of Africa.  They were all trying to get to China and India’s lucrative trade routes. Basically, within 30 to 50 years, the Spanish had turned this entire region into a Spanish lake, controlling the main ports and only allowing Spanish ships. 

The Portuguese did the same thing. Alfonso was responsible for the blue region. It only took them six years or so, and involved defeating the Ottoman Empire in several naval battles, conquering various port cities around the edge, defeating some Indian empires, and so forth.

Here is the map of Portuguese territory in India which blew my mind. 

I didn't realize that they had conquered so much so quickly. I haven't looked up this exactly, but it took many more years to conquer all of this territory. 

When I usually think of colonialism in India, I think of the British. But according to this map, the Portuguese made quite a lot of conquests themselves before then. Anyway, the advantages were not god-like, in my opinion. I go into more detail in the post, but it was not like they had nanobots or any other super weapons in which there were no defenses against. 

They also didn’t have machine guns, or even normal guns. They had extremely ancient arquebuses that took two minutes to reload after you fired them. Most of the time, they just used swords for their fighting.. I could go on.

I do think that they won because of their technology, but it definitely was not  a god-like advantage.  So I think this disrupts the simple model that I mentioned earlier. And it's not like they just had more strength than the regions that they conquered either. 

Or if we do say that they had more strength than the regions that they conquered, it's not because they had some sort of god-like advantage. Here's my model which I think better describes how these, and possibly many other, conquests worked.

Military takeover is a diplomatic process in which you convince people to obey you. You do that by offering  a better Schelling point than rival claimants to the throne, and by defeating your rivals in battle.  You don't need to defeat them all at once; you can start small and scale up.

Each victory gets you more allies which you can deploy in the next battle. Rivals can become allies after, or even without, a fight. This is how these conquests worked. It wasn't like Pizarro showed up at the Incan Empire’s borders and was met by a massive army, which he then defeated. Same thing for Cortes. There were  a series of episodes in which the conquistadors incrementally gained more power. 

If there's an AI takeover, maybe  the AI will have god-like powers with  nanobots and other powerful technologies. But it is also entirely possible that there could be an AI takeover without these things. Thank you.

 

Questions

Ben Pace: Thank you very much, Daniel. That was fascinating and you crammed quite a lot into it. I have some questions but so does everyone else. Daniel Filan, would you like to ask the first question?

Daniel Filan: Yes. This is  a combination of comment and question.

 Thinking about god-like powers, I recently read a piece about  Cortes taking over the Aztec Empire, and apparently a really big advantage was armor impervious to the kinds of weapons that Aztecs had. If other people can’t hurt you, then it is, in a sense, a god-like power. What I am wondering is to what extent they had the god-like power of ratcheting, where the conquistadors couldn’t go backwards or get hurt while still being able to hurt the Aztecs.

Daniel Kokotajlo: I think that fits, to some extent, with the story of Cortes, which I've now read in great detail. I think that the post that you mentioned, exaggerates the extent to which the armor was superior.

For one thing, a lot of the Spanish gave up their metal armor and took up a Aztec cloth armor instead because it was more comfortable and  almost as good, which suggests that their metal armor wasn't superior. Also, plenty of conquistadors were killed in battle. I think that a lot of their success had to do with their tactics rather than  just the armor. That said, armor did likely play a large part in their victory.  It is wrong, however, to conclude that they were invincible because of it when in fact they came close to being wiped out on several occasions. 

Ben Pace: John, would you like to ask the next question?

johnswentworth: Let's say, hypothetically, that I'm planning to take over the world through technological means, what sort of technology should I be focusing on in order to get the most bang for my buck?

Daniel Kokotajlo: I think that realistically, an AI takeover would mostly not be military.

johnswentworth: I didn't ask about AI takeover.

Daniel Kokotajlo: Okay, so I have a list of technologies that I think would be the modern equivalents of the armor.

Maybe I'll draft  a blog post sometime about my own speculations on technologies that aren't that far ahead into the future, but could be pretty powerful. That's what I really should have said in response to Daniel's question. That yes, the armor was really consequential  in some sense.

It enabled the conquistadors to  go really far in battle and in politics. But it wasn't fundamentally different from  what the Aztecs had. It was  similar but better. 

And similarly, I think that an AI designing better guns,  armor, and coordination technologies  for soldiers could end up being extremely advantageous militarily, even if it's not something fundamentally new, like nanobots. Even if it is something simple like aim assist for soldiers’ guns or steering assist for vehicles, these could provide a massive military advantage. 

Ben Pace: Go on. Throw us a bone. Give us one example of a technology, especially a slightly more surprising one. We've heard of guns.

Daniel Kokotajlo: Sure. Aim assists for infantry rifles would be really beneficial . The US military is currently looking into this. But a simple version could just be  a camera hooked up to a  gun and trigger so that when you activate it, it fires the bullet when it calculates that the bullet will actually hit the target.

You wouldn’t  have to aim carefully, you could just wave the gun in front of the target and it shoots at exactly the right moment.. A more sophisticated version of this, which I have thought of, but not sure if anyone else has f, is having some sort of cold gas thruster on the tip of the gun.

Or maybe a gyroscope  allowing the gun to move itself although it sounds more complicated. This doesn't sound like much, but I think it would probably be a pretty big deal on the battlefield. For one thing, you wouldn't have to poke your head out of cover to shoot at the enemy.

You could just poke the gun out and it would do the shooting  I also think being substantially more accurate would make cover fire a lot more effective. And that could allow  you to be much more mobile on the battlefield. Anyhow, this is just one example.

Ben Pace: It is at this point, I realize I've really not actually tried to figure out how I would use modern tech to improve military practices. And I am a bit scared now. All right, the next questioner is orthonormal.

orthonormal: Are there any known examples of attempted conquistadors that failed?

Daniel Kokotajlo: That's a really good question. There's probably a half dozen to a dozen parties that wandered around the Americas and didn't really find much and then died of starvation or something.

In fact, that very nearly happens to Pizarro and Cortes. There weren't any other major empires like the Incas or the Aztecs in the Americas. As for the rest of the world, I don't know the full answer to that question. There were likely  attempts to conquer India that failed before eventually succeeding. 

53