I intend to refute a commonly held idea, which is that social instability secondary to large-scale technological unemployment will necessarily result in a new social contract where those who control AGI[1] take the interests of the newly minted precariat into consideration, for example with some form of universal income.
Argument from Historical Precedent
I see that this line of thinking originates in historical examples involving social stress reaching a tipping point, whereupon a determined group of sufficient size effectively swarms the ruling minority and acts with the intention to produce a new equilibrium that they judge more favourably.
Liberal democracy is both the product of revolution and a uniquely stable equilibrium. Its stability can be explained by it giving everyone an ostensibly equal share in determining the direction of governance[2], thus theoretically incentivizing the population at large to protect it from anything which would lead to them losing their share.
Swarming as Leverage
The basic strategy of a revolution leverages an overwhelming numerical advantage to overcome the coercion produced by an existing power structure.
To paint an extreme example, an unarmed fraction of the revolution is sacrificed to absorb a soldier's bullets, while the remainder swarms him before he has a chance to reload. Now they're no longer unarmed.
The effectiveness of such a movement compounds once the other guardians of the incumbent see their own position as untenable, and people on the inside start defecting on the original power structure to make it in a post-revolutionary equilibrium, thus turning the revolution into a self-fulfilling prophecy.
The Erosion of Leverage
Importantly, humans swarming a power structure doesn't work anymore in a post-AGI world.
Your union threatens to strike, and your boss can now say "goodbye" to all members without worrying about cuts to production.
You shoot down one slaughterbot, and a hundred more are deployed in the same instant.
Potential threats are neutralized before they have any chance of materializing.
The Fate of Democracy post-AGI
"But doesn't democracy fix this?", you might ask. "Won't we be able to just vote our way into a desirable equilibrium?"
Let's assume this works, and you now live under some kind of functional UBI system.
What do you do if your post-AGI state stops being a democracy? The present-day answer is: you stand up to save it, knowing that a post-democratic equilibrium is predictably unfavourable to the majority, especially as the incoming power structure tries to protect itself using e.g. mass surveillance or by terrorizing the population into submission.
Their strategy is geared toward lowering the probability that a critical mass of people rises up and overthrows them. In a post-AGI world, there is potentially no critical mass of people that can overwhelm the system. The likelihood that you succeed at dislodging an unjust power structure goes to zero.
With your ability to force a favourable equilibrium by means of collective action gone, and your expected utility to the system also gone because your labour value went to zero, what are the incentives to continue giving you your UBI, or to otherwise entertain your continued existence?
Conclusion
Going back to the title, it is clear that the population's satisfaction is still a meaningful check on governance today, just as it has been in the past. Different states, driven by different ideologies, find different equilibria which are more or less satisfactory for the majority of people.
However, given current trends, it seems that popular satisfaction could soon become completely optional from the perspective of the power structure for the first time in human history, and if you don't like the outcome, then it's not like you'll be able to revolt your way out of it anymore.
"Why assume such alignment?", you might ask. The answer is that the economic incentive structures pushing the AI frontier are rolling with this assumption, which ends up producing AGI regardless of whether it is justified or not.
All the ways in which it just ain't so (e.g. large campaign donors, media manipulation) appear to be too opaque to raise enough interest in creating a new equilibrium.
I intend to refute a commonly held idea, which is that social instability secondary to large-scale technological unemployment will necessarily result in a new social contract where those who control AGI[1] take the interests of the newly minted precariat into consideration, for example with some form of universal income.
Argument from Historical Precedent
I see that this line of thinking originates in historical examples involving social stress reaching a tipping point, whereupon a determined group of sufficient size effectively swarms the ruling minority and acts with the intention to produce a new equilibrium that they judge more favourably.
Liberal democracy is both the product of revolution and a uniquely stable equilibrium. Its stability can be explained by it giving everyone an ostensibly equal share in determining the direction of governance[2], thus theoretically incentivizing the population at large to protect it from anything which would lead to them losing their share.
Swarming as Leverage
The basic strategy of a revolution leverages an overwhelming numerical advantage to overcome the coercion produced by an existing power structure.
To paint an extreme example, an unarmed fraction of the revolution is sacrificed to absorb a soldier's bullets, while the remainder swarms him before he has a chance to reload. Now they're no longer unarmed.
The effectiveness of such a movement compounds once the other guardians of the incumbent see their own position as untenable, and people on the inside start defecting on the original power structure to make it in a post-revolutionary equilibrium, thus turning the revolution into a self-fulfilling prophecy.
The Erosion of Leverage
Importantly, humans swarming a power structure doesn't work anymore in a post-AGI world.
Your union threatens to strike, and your boss can now say "goodbye" to all members without worrying about cuts to production.
You shoot down one slaughterbot, and a hundred more are deployed in the same instant.
Potential threats are neutralized before they have any chance of materializing.
The Fate of Democracy post-AGI
"But doesn't democracy fix this?", you might ask. "Won't we be able to just vote our way into a desirable equilibrium?"
Let's assume this works, and you now live under some kind of functional UBI system.
What do you do if your post-AGI state stops being a democracy? The present-day answer is: you stand up to save it, knowing that a post-democratic equilibrium is predictably unfavourable to the majority, especially as the incoming power structure tries to protect itself using e.g. mass surveillance or by terrorizing the population into submission.
Their strategy is geared toward lowering the probability that a critical mass of people rises up and overthrows them. In a post-AGI world, there is potentially no critical mass of people that can overwhelm the system. The likelihood that you succeed at dislodging an unjust power structure goes to zero.
With your ability to force a favourable equilibrium by means of collective action gone, and your expected utility to the system also gone because your labour value went to zero, what are the incentives to continue giving you your UBI, or to otherwise entertain your continued existence?
Conclusion
Going back to the title, it is clear that the population's satisfaction is still a meaningful check on governance today, just as it has been in the past. Different states, driven by different ideologies, find different equilibria which are more or less satisfactory for the majority of people.
However, given current trends, it seems that popular satisfaction could soon become completely optional from the perspective of the power structure for the first time in human history, and if you don't like the outcome, then it's not like you'll be able to revolt your way out of it anymore.
"Why assume such alignment?", you might ask. The answer is that the economic incentive structures pushing the AI frontier are rolling with this assumption, which ends up producing AGI regardless of whether it is justified or not.
All the ways in which it just ain't so (e.g. large campaign donors, media manipulation) appear to be too opaque to raise enough interest in creating a new equilibrium.