I do not think you are necessarily wrong in general, but this is part of a much larger class of concerns about "lock-in" and path dependency after ASI is achieved. I mean, it's supposed to be "the last invention" : a lot of arrangements will be hard or impossible to change after ASI is achieved, including political arrangements. I think that's why the field of alignment as a whole is concerned about making sure things are right before ASI is achieved. Murderbots being on the table against political opposition counts as a form of misalignment.
I think you are just wrong about swarming. Revolutions do not work by zerg rushing since... the invention of grapeshot, at least. Information cascades can and have led to regime change from a very small group of committed activists, if the legitimacy of the regime is shaky enough.
That being said, the deeper point stands, and I agree that ASI would obliterate most modes of political actions viable today via labor power and the implicit threat of regime change, including, if misaligned enough (totalitarian surveillance+murderbots), information cascades. More broadly, I haven't seen enough discussion of what the political economics of the post-ASI world should look like and how to actually get there. Again, once arrangements have congealed in place, it will be probably very difficult to change.
I think you are just wrong about swarming. Revolutions do not work by zerg rushing since... the invention of grapeshot, at least. Information cascades can and have led to regime change from a very small group of committed activists, if the legitimacy of the regime is shaky enough.
I don't know about "zerg rushing", but some recent examples of popular pro-democracy revolutions are the Fall of the Berlin Wall and the collapse of communism in Eastern Europe in 1989-1991, and Ukraine's two revolutions in 2005 and 2014. Interesting point about information cascades, perhaps a more accurate model is (number * commitment relative to the regime), though size still matters to the extent that the activist group can project legitimacy and power, and not worry about getting swarmed in turn.
I actually believe there is a good chance that political economy might get washed away together with all other human-era concepts by Actually Something Incomprehensible. The dominant narrative ("Everything will be ok, UBI, right?") is wishful thinking, obscuring the fact that being economically obsolete and unable to constrain the system by threatening revolution puts the vast majority of people[1] in a genuinely precarious position.
- ^
or everyone if one accepts the added assumption that ASI has no reason to accept human property relations.
In these revolutions, the disintegration of state apparatus loyalty was a key factor, with poorly organized groups primarily serving to undermine the legitimacy of the existing regime and facilitate coups within the government, as seen in the Bangladeshi Revolution or the Ukrainian Revolution.
However, history records at least two distinct revolutionary models: In the October Revolution, the Bolsheviks exploited informational advantages and a vanguard-organized military to execute a decapitation strike and seize control of the existing state apparatus. In China's revolution, the Communist Party established its own state apparatus capable of delivering more effective governance than the original government, ultimately defeating the state machinery of warlords and the Kuomintang through warfare.
Unfortunately, in the post-AGI era, neither of these methods of launching revolutions is any more likely to succeed than expecting a coup within the government.
I think the framing of the "October Revolution" as an obvious turning point is mostly a retrofit by Soviet historiographers. It was a coup which lead to several years of civil war with the incumbent and other actors, ultimately favouring the Bolsheviks. I understand that civil war was also the primary arena that determined regime change in China.
I agree that these differ from the other revolutions mentioned previously, though I'm personally more inclined to put them in a different conceptual bucket altogether, noting that them calling themselves revolutions shares a resemblance with North Korea calling itself a democratic people's republic in that the choice of terminology appears to be deliberately self-legitimizing.
With that said, I agree that both of these ways to engender regime change encounter the same predictable failure mode post-AGI.
I agree. AI optimists like Kurzweil usually minimize the socio-political challenges. They acknowledge equality concerns in theory, but hope that abundance will leverage them in practice (if your share is only a little planet that's more than enough to satisfy your needs). But a less optimistic scenario would be that the vast majority of the population would be entirely left behind, subjected to the fate that knew horses in Europe and USA after WWI. May be some little sample of pre-AI humans could be kept in a reserve for curiosity, as long as they're not too annoying, but it's a huge leap of faith to hope that the powerful will be charitable.
In reality, I think that it's unlikely that a self-actualizing AI system of sufficient caliber will necessarily respect contemporary property relations, for basically the same reason that a hypothetical post-AGI economical elite doesn't necessarily support the underclass: it doesn't have to.
I just want to throw a wrench in the "everyone will necessarily be taken care of" narrative, which is promulgated with an entirely unwarranted degree of confidence by people who (superficially) benefit from everyone believing it to be true.
The basic strategy of a revolution leverages an overwhelming numerical advantage to overcome the coercion produced by an existing power structure.
No it does not. Can you name one successful revolution where the revolutionaries had an "overwhelming numerical advantage"? In the American, French and Bolshevik revolutions the revolutionaries, at best, had parity with the forces opposing them. They succeeded not because they had overwhelming numerical advantage, but because they had advantages in coordination and cohesion that enabled them to strike while their opponents were still preparing.
1
1In the American, French and Bolshevik revolutions the revolutionaries, at best, had parity with the forces opposing them. They succeeded not because they had overwhelming numerical advantage, but because they had advantages in coordination and cohesion that enabled them to strike while their opponents were still preparing.
This claim, as it stands, is false. If a thousand Frenchmen vs. a hundred soldiers of the royal government defending the Bastille isn't numerical superiority... I'm not interested in derailing into historical debate here; ask your favourite LLM for a fact check on the quoted portion.
I will say one thing, which is that the notion of "striking while their opponents are still preparing" actually smuggles in an assumption of numerical superiority[1], and your point about "cohesion" is the morale differential that contributes to revolutions becoming self-fulfilling prophecies.
The point is that you will not be able to coordinate your way out of an AGI dystopia regardless of your numerical superiority or inferiority. Once we're past the event horizon, what leverage do think you have left in a system that doesn't need you for anything, that can crush you like a bug? Well, surely once enough people agree that Shit Sucks, perhaps because the UBI checks started to dry up without explanation, we can all... We can all just... Wait, what's that buzzing sound?
- ^
Alice has an army of 100 soldiers, 10 of whom show up to the fight while 90 are "still preparing" (or on the other side of the Empire, or dragging their feet because they don't care that much about defending Alice, are scared of Bob or secretly think he has a point). Bob has an army of 50 soldiers, 40 of whom show up to the fight (because they very much care about defeating Alice). Who has the numerical advantage?
A thousand Frenchmen may have achieved local overmatch against the forces of the regime. But at that time, the Revolutionary forces most assuredly did not outnumber the entirety of the French military.
Who has the numerical advantage?
Alice, of course. Bob's army wins via superior cohesion and morale. By your logic there can be no victory against numerical odds, because you can always look closely and find specific points where the victorious side had numerical overmatch against the losing side.
I say that Bob has a 4:1 advantage. Yes, it's "only" at the moment of contact, however this is what produces anything of consequence and thus the only thing that matters for the outcome.
The fact that Alice is unable to mobilize her entire army means that her apparent 2:1 advantage is fake in any practical sense, a paper tiger.
Though I'm sure that Bob's historiographers would (instrumentally) agree with you because the underdog story presents a more compelling, better self-serving narrative.
You're free to use your own idiosyncratic definition of "numerical superiority", but then you shouldn't be surprised when your interlocutors are persistently confused as to how you can claim an army of 1000 has "numerical superiority" against an army ten times its size.
"I don't understand!", said commander Alice as her palace got surrounded. "There were so many more of us than there were of you". "Were there really, or was the number on your spreadsheet a fiction?", said Bob. "If they don't show up to the fight and switch over to my side en masse, why were they included in your troop count? In what way were they yours?"
Here's another example. At first glance, it looks like black should win the game easily due to the apparent points differential at the start. But what meaning does "point superiority" have when the game starts and black's pieces don't respond to commands and start switching their colour in a cascading fashion, resulting in white's victory?
These are fictional examples. In reality, like at the Battle of Cannae, for example, the Carthaginian force, led by Hannibal Barca, was outnumbered by the Romans by nearly 2:1 (roughly 40,000 Carthaginians against around 80,000 Romans). And yet the Carthaginians won by trapping the Romans in a double-envelopment and attacking them from all sides, preventing the Romans from concentrating their superior numbers, allowing Hannibal's forces to slaughter the Romans. Although Hannibal's forces may have outnumbered the Romans at the actual line of contact, no one claims that the Carthaginians had numerical superiority at Cannae.
Similarly, at Agincourt, the English were outnumbered by the French by approximately 2:1. The French were defeated because the English used terrain cleverly, forcing the French forces to attack down a narrow path surrounded by forest on both sides. This, combined with the superior firepower of English longbows, enabled the English to decimate the French, and win a major victory in the Hundred Years War. Once again, although the funnel effect of the terrain may have meant that the English enjoyed numerical superiority at the point of contact, Agincourt is widely regarded as an example of an underdog victory, where a smaller English army won a major victory against the odds against a much larger French foe.
At the battle of Narva, Charles XII of Sweden used a snowstorm to hide the approach of his 10,000 troops, suprising the 35,000 Russians laying siege to Narva and putting them to rout. Do you claim that the Swedes had numerical superiority over the Russians merely because some of the Russian troops could not see the Swedes until it was too late?
At the Battle of Trafalgar, Nelson had 27 British ships, against Villenueve's combined Franco-Spanish fleet of 33. However, Nelson was able to maneuver his fleet such that it split Villenueve's line into thirds, allowing his fleet to defeat Villenueve piecemeal. Although Nelson had numerical superiority at specific, crucial points in the battle, overall Villenueve's fleet was larger, and Trafalgar is seen as a victory of British elan and seamanship, not numerical superiority.
The reason I bring up all these battles is because numerical superiority is but one part of victory, and, arguably, not even the most important part. Logistics, terrain, morale, and tactics all have parts to play as well. Redefining numerical superiority as you've done elides those factors, as you can always say, "Well, actually, at the point of contact, the victorious forces had numerical superiority over the vanquished." That's true, but it's in some ways as much of a tautology as, "The team that wins the game is the one that scores the most points." It doesn't inform any predictions about the future, nor does it indicate when you should be surprised by an outcome.
I appreciate the historical research, but insofar as all of your examples are of interstate conflicts, we have diverged far from the original context of revolutionary calculus.
The key differentiator is that the incumbent's army is not "vanquished" by a successful revolution; it is absorbed by it after an internal transfer of political legitimacy away from the incumbent and toward itself.
This process can be bloodless. The fall of the GDR comes to mind, where mass demonstrations produced their own political legitimacy with sheer numbers, causing automatic restraint from security forces despite the appearance that on paper, they (in aggregate) outnumbered the demonstrators; and the fact that they outgunned and basically outmatched them on every tactically relevant metric.
Returning to the original point of this post, this ultimately human mechanism whereby security forces hesitate when faced with a popular uprising of sufficient size (e.g. "there are so many, and they're not scared, maybe they're right", or "maybe some of my neighbours/friends/relatives agree with them or have joined in") can cease to exist in a regime where advanced AI is in charge of securing the state.
You and everybody else lose the two levers you could historically use to protest the worst indignities and abuses of power the state can subject you to: the lever of withholding your labour in a general strike (because human labour stops being a factor of production), and the lever of participating in a political revolution (because no critical mass of people can overwhelm the system, which loses nothing if you die). The disempowerment of the people is a predictable consequence of the current trajectory.
That's true, but it's in some ways as much of a tautology as, "The team that wins the game is the one that scores the most points."
That's not a tautology, and indeed there are games where the opposite is true, such as golf.
I intend to refute a commonly held idea, which is that social instability secondary to large-scale technological unemployment will necessarily result in a new social contract where those who control AGI[1] take the interests of the newly minted precariat into consideration, for example with some form of universal income.
Argument from Historical Precedent
I see that this line of thinking originates in historical examples involving social stress reaching a tipping point, whereupon a determined group of sufficient size effectively swarms the ruling minority and acts with the intention to produce a new equilibrium that they judge more favourably.
Liberal democracy is both the product of revolution and a uniquely stable equilibrium. Its stability can be explained by it giving everyone an ostensibly equal share in determining the direction of governance[2], thus theoretically incentivizing the population at large to protect it from anything which would lead to them losing their share.
Swarming as Leverage
The basic strategy of a revolution leverages an overwhelming numerical advantage to overcome the coercion produced by an existing power structure.
To paint an extreme example, an unarmed fraction of the revolution is sacrificed to absorb a soldier's bullets, while the remainder swarms him before he has a chance to reload. Now they're no longer unarmed.
The effectiveness of such a movement compounds once the other guardians of the incumbent see their own position as untenable, and people on the inside start defecting on the original power structure to make it in a post-revolutionary equilibrium, thus turning the revolution into a self-fulfilling prophecy.
The Erosion of Leverage
Importantly, humans swarming a power structure doesn't work anymore in a post-AGI world.
Your union threatens to strike, and your boss can now say "goodbye" to all members without worrying about cuts to production.
You shoot down one slaughterbot, and a hundred more are deployed in the same instant.
Potential threats are neutralized before they have any chance of materializing.
The Fate of Democracy post-AGI
"But doesn't democracy fix this?", you might ask. "Won't we be able to just vote our way into a desirable equilibrium?"
Let's assume this works, and you now live under some kind of functional UBI system.
What do you do if your post-AGI state stops being a democracy? The present-day answer is: you stand up to save it, knowing that a post-democratic equilibrium is predictably unfavourable to the majority, especially as the incoming power structure tries to protect itself using e.g. mass surveillance or by terrorizing the population into submission.
Their strategy is geared toward lowering the probability that a critical mass of people rises up and overthrows them. In a post-AGI world, there is potentially no critical mass of people that can overwhelm the system. The likelihood that you succeed at dislodging an unjust power structure goes to zero.
With your ability to force a favourable equilibrium by means of collective action gone, and your expected utility to the system also gone because your labour value went to zero, what are the incentives to continue giving you your UBI, or to otherwise entertain your continued existence?
Conclusion
Going back to the title, it is clear that the population's satisfaction is still a meaningful check on governance today, just as it has been in the past. Different states, driven by different ideologies, find different equilibria which are more or less satisfactory for the majority of people.
However, given current trends, it seems that popular satisfaction could soon become completely optional from the perspective of the power structure for the first time in human history, and if you don't like the outcome, then it's not like you'll be able to revolt your way out of it anymore.
"Why assume such alignment?", you might ask. The answer is that the economic incentive structures pushing the AI frontier are rolling with this assumption, which ends up producing AGI regardless of whether it is justified or not.
All the ways in which it just ain't so (e.g. large campaign donors, media manipulation) appear to be too opaque to raise enough interest in creating a new equilibrium.