I was happy to see this post. Would strong-upvote it ten times if I could. Thank you.
Between this, the burger post that you linked, and the Torment Nexus post, there's a strong argument why people shouldn't build a personal raft for themselves while leaving the majority to sink: it's evil and it won't even work. People should work on something that can be credibly described as building a raft for everyone, or failing that, shouldn't work on AI at all.
The next question is what counts as building a raft for everyone. Political action in favor of regulation? (I'd say yes.) Working on alignment at big labs? (I'd say no.) Working on open models to tilt the balance of AI power toward the masses? (I'd say yes.) I realize these questions are controversial and my position in particular is pretty minority. But in any case I think these are the right followup questions.
The whole discourse around a “permanent underclass” always seemed somewhat farcical to me — at best a distraction, at worst an actively harmful meme insofar as it freaks people out and tries to provide (shallow, but nevertheless) justification going whole hog on building strong AI. So it has been with a sense of dismay that I’ve seen this phrase come into popular parlance 1 2 3, and increasingly come to motivate a sort of frenzied upward striving in my second- and third-degree connections. Among those I know, the standard framing is that we need to “get the bag” while there’s still a chance, to take advantage of these last few years of income-potential to escape whatever horrible fate lays in store for the rest of humanity. I think that events are unlikely to play out this way; even setting aside “doom” arguments (for reasons I’ll get to below), I think history shows that in times of transition, wealth is far less of a guarantee than people intuitively think. So “the bag” will provide little guarantee of thriving, or even surviving, into a “post-AI world”.
There have certainly been arguments against the “grab the bag” thesis (as I’ll henceforth refer to it) before, but my experience they either tend to A) go hard in one direction by arguing that Things Will Be Okay actually (e.g. Elon Musk will be so wealthy that he’ll give you a Universal High Income simply out of the goodness of his heart), or B) go hard the other way and lean too much on predictions of AI doom. I think (A) is wrong; to pick a choice quote from elsewhere:
I do not see any compelling reasons for this state of affairs to change just because wealth increases by another 10x or 100x.
With (B), on the other hand, I actually agree, but it’s a hard argument to make: many people are at this point well-trained to either shut it down entirely, or even in the case that they do half-accept it, still cling to the idea that money will somehow save them. To be clear, I think that the seed of the concept does ring true: I do expect to see returns on capital spike, and probably for a time we will indeed see some great unhappy underclass emerge. What doesn’t make sense is how exactly the capitalists would stabilize the resulting society — i.e., how can they secure for themselves the role of “permanent” ruling elite over this “permanent” underclass? Without the former (to protect private property rights, to maintain a monopoly on violence, to keep the GPU clusters running), the rules of the game go out the window, and the “underclass” dynamic becomes irrelevant.
So what I hope to do here is put forward an argument against the “grab the bag” thesis, as I’ll henceforth refer to it, that can stand on its own without relying too much on the doom-argument. I’ll first motivate my argument with a gesture towards past examples of historical transition, then try to ground it in a list of possible futures that between them exclude “grab the bag” as a viable path to safety.
Who is the Modern Handloom Weaver?
What has since evolved into this post first originated as a comment on Fabricated Knowledge’s Engel’s Pause and the Permanent Underclass. That article heavily leaned into a historical analogy with the industrial revolution, using the development of the industrial proletariat as a sort of ‘existence proof’ for the creation of a politically disempowered class with dramatically worse QoL and a bevy of unpleasant impositions.
In O’Laughlin’s analysis, the role of handloom weavers will today be filled by what he calls the “Information Artisan Class”:
I think this is a fine analysis, at least on its face, and I don’t have anything against this kind of historical analogy. In particular, I probably ascribe significantly more explanatory power to past examples of societal change than most people on this forum: in my opinion, while the rate of change has certainly accelerated, many of the underlying dynamics remain intact, and so we can learn a lot through analogy. However, I would argue that the specific scenario O'Laughlin chose is actually not the right one to consider. This is because the industrial proletariat, immiserated as they were, never formed a “permanent underclass”, at least not within the economic core of the developing West. The political power they were able to accrue through unions, strike action, and militant activity allowed them to either demand better conditions and guarantees from both the owner-class and politicians (as seen in the New Deal, or in European social democracies) or revolt alongside the increasingly-fraught peasantry and disempower the capitalists entirely (as in the USSR, China, and elsewhere — regardless of what you think about their governments thereafter, I think it’s safe to say that the former capitalists were not in charge) — all within the span of less than a century, and for the revolutionary examples really only but a few decades.
Yet we don’t have to look far to find a much better example of a ‘permanent underclass’ in history: indeed, the very predecessors of that industrial proletariat, i.e. the medieval peasant who, once pushed off their farms, came to man the mills which would eventually replace the artisanal class.
The Medieval Peasantry and the Senatorial Order
The typical Western history curriculum spends a good deal of time on Rome, a good deal of time on Medieval Europe, and very little time on the period between. To some extent this is understandable, as the period was (definitionally) one of rapid and chaotic change. Unfortunately for us, born in these increasingly rapid-and-chaotic times, this is the most apt analogy for the moment. To call one particular gap to mind, try thinking back on how ‘the people’ were portrayed in each of the above. In the former, we are told ‘the people’ were citizens, with voting rights, protection under the law, and a (relative) breadth of economic opportunity. In the latter, we are told of an unfree peasantry, many little better than slaves, and all of them subjugated to the whims of their aristocratic overlords. Both images are too reductive by far (e.g. the Roman plebs had far less power than “voting” would suggest, and medieval law codes gave far more rights to the medieval peasantry than the popular image suggests) — but there is nevertheless a real gap. In one instance there are free, albeit often poor, citizens, and in another there are serfs, bound to the land and under the hand of their local gentry. How do we get from A to B?
A naive view would be that the aristocrats, having the horses and the swords, simply took what they wanted. The first counterpoint to this is that not all peasants were serfs: while they were all doing largely the same work, and would often both pay rent to the same landlord-nobleman, the latter were bound to the land, subject to additional taxes and duties, and required by law to pay subservience to their lords in a way the former were not. In medieval history, the distinction between ‘free’ and ‘unfree’ peasant was relatively well-regulated by law — at least in western Europe, peasants rarely went from the former to the latter, and e.g. in England there was in fact a steady trend of emancipation where unfree villeins would purchase their freedom, even at great material cost. Free peasants could even hope to, over the course of generations, grow their holdings and, if fortunate, ‘graduate’ to the ranks of minor gentry through pretensions at higher status or aristocratic lineage. The existence and growth of this free peasantry makes it clear that the explanation cannot be as simple as “aristocrats got what they wanted”.
Rather, the unfree serf can be understood as a product of the preceding period of transition, emerging out of Late Antiquity and essentially ‘bequeathed’ unto the subsequent Early Medieval world. The initial populations of, say, 200BC were slowly converted over the course of the intervening centuries through a combination of economic pressure, legal imposition, and ultimately societal collapse, such that by say, 600AD, they looked more-or-less like the peasants we see in the history books. In its first centuries of expansion, the Roman polity acquired new territory through conquest, creating many new free smallholders (both by integrating local populations and settling veterans in the conquered territory), while simultaneously bringing in a great number of slaves. This engine, of slave-based plantation agriculture, enabled the senatorial elite to reliably out-compete their citizen-smallholder neighbors in both economies of scale and ‘cost of labor’, allowing them to acquire more land and thus accrue larger and larger shares of wealth. In time (though at different rates in different places) many smallholders became dependent on their large-landholder neighbors, e.g. by taking out loans during hard times, or because they were made to sell their land to those larger neighbors and lease it back for a fixed rent. Thus impoverished and supplicated through these bonds of debt/patronage, their status already resembled that of the later ‘unfree peasant’ in many ways — yet they still retained freedom of movement and thus the ability to decide their own circumstances.
As the economy of the empire began to get on the rocks, however, Roman emperors increasingly sought to immobilize workers to ensure that the land remained worked and prices remained stable. This program reached its height under Diocletian, a former military officer who successfully put the empire back together after it very nearly fell apart during the Military Anarchy of the 3rd century. Using his newly-stable base of power, he sought to re-stabilize Roman society by legal fiat, imposing regulations to forestall any further disintegration. You are likely familiar with his Edict on Prices, which attempted (unsuccessfully) to set price caps on goods throughout the Empire, but the same set of reforms also initiated what modern historiography terms the “freezing of professions”, where roles and occupations were made fixed and hereditary — the son of a soldier would be a soldier, the son of a city counselor a counselor, and the son of an artisan an artisan. The goal was to ensure predictability for the Empire itself: rather than volatile semi-market-based supply chains and labor movement between them, the relations of production were formalized through administrative-military fiat. A few civil wars later and we see this concept mature further under Constantine, who formally enshrined the status of “coloni adscripticii” in law: unlanded laborers were thus made “slaves of the land”, in Roman legal parlance, without the right to leave it and subject to discipline from their landlords should they try. Not all ‘peasants’ fell into this category, creating the distinction between free and unfree that would persist through the medieval period. Thus we can observe the emergence of the sort of dependent, immobile labor relations that would ultimately characterize the medieval ‘permanent underclass’.
But where did the senators go?
Over the same period as the free-ish citizens of the Roman empire were replaced with the peasants-and-serfs of later centuries, you might notice another group ‘changing place’ alongside them. Above the feudal peasant was the feudal aristocrat; above the Roman citizen was a Roman senator. And note well that the aristocrats of latter years were (generally) not the same as the senators of yore — the first King of the English was not named “Tiberius” or “Julius”, but Æthelstan. So even as the senatorial elite succeeded in subjugating and more-totally exploiting the once-unruly free citizens beneath them, they themselves disappeared before they could enjoy the spoils.
In sweeping terms again, I will reach to say that by the 8th century no clearly identifiable senatorial order remained as the dominant ruling class in (more or less) any part of what had once been the western Roman Empire. Britain went first, with the economy suffering “radical material simplification” in the century after the withdrawal of the garrisons, leaving post-Roman Britain to fight a losing battle against waves of Pictish and Germanic invaders. Africa suffered a similar fate, but with a rather more violent course as first the Vandals, then the resurgent Romans under Justinian, then the Umayyads under Abd al-Malik successively seized the territory and dispossessed many of its inhabitants. Both Italy and Hispania saw a brief swan-song of collaboration between the old Roman elite and the new Germanic aristocracy, but with the Gothic war in the former and the Umayyad conquest of the latter, the power of the senatorial elite was largely broken, and new centers of power developed in both — c.f. the struggles between Lombard dukes and Catholic bishops that characterized much of the succeeding years of Italian history. Gaul had perhaps the highest degree of long-term continuity, but here as in Italy, the new order that emerged was still one rooted in military offices and ecclesiastical hierarchies, with the new Germanic military elite firmly in charge, and the old senatorial order largely integrating thereinto.
This was not a “new boss, same as the old boss” situation, either: the new order was different culturally (often to the exclusion and discomfort of the old) as well as operationally. For the former point, we have considerable literary evidence from the old order, e.g. Cassiodorus’ Varieae, my personal favorite, which conveys with underlying sadness how the old markers of status, e.g. erudition and rhetorical skill, had no place in the dining-halls of the Gothic kings. So too did the practical day-to-day systems of government quickly became unrecognizable. The ‘illegible Carolingian army’ had almost nothing in common with the Roman legions of old, and the 843 partitions at Verdun (infamous for first demarcating the battle-lines between today’s France and Germany) embodied a particularly Germanic concept of partible inheritance that stands in stark contrast to the former Roman concepts of unitary imperial power. So while in all cases there was at least a degree intermingling, in none did the old Roman elite stay in the driver’s seat — at best they changed to fit the new order, at worst they were replaced outright.
What is my point here? Despite “grabbing the bag” to an incredible degree, the Western senatorial order mostly failed to persist as the ruling class, even where individuals or families survived by adaptation.
Who is the Modern Roman Senator?
So on one hand, the senatorial elite “won”, in subjugating their domestic class-enemies and reducing them to a sort of “permanent underclass”. But on the other, they largely did not survive (as a class, even if the individuals were not wiped out) to enjoy the fruits of this victory. How did this happen? In the erstwhile political order, the “decisionmakers” of society were also its defenders, forming a class of citizen-soldiers that sat between the senatorial elite and the broader, poorer population. The new system that followed it relied on the loyalty of exogenous military auxiliaries to enforce the framework that allowed for the production and extraction of wealth. Through the civil wars that empowered Caesar and then Augustus, and later the reconstitution of the empire under Diocletian, this military was what made it possible to enforce serfdom through law — and yet, this class of military auxiliaries was also the same one that ultimately undid the senatorial order. Forgive the sweeping gesture, but I want to point at a thematic through-line: not necessarily a single mechanism, but at the very least a recurring theme. From the civil wars of the late republic through the militarized Flavian dynasty, in the praetorian palace coups and the usurpations of the Military Anarchy, to ultimately the collapse in the west, the brief swan-song under Theodoric, and the emergence of new feudal aristocracies, we see a broad trend emerge. First a ‘defanging’ of the economic elite, then slow migration of decision-making authority to their nominal military subordinates, and finally the elimination of their class in favor of those same ‘subordinates’. Today our economic elite are already defanged; it is not too difficult to imagine how we might move further along the same track. In medieval Europe, this ended with the creation of a new feudal structure where those who controlled economic production simultaneously held the military power which buttressed it. This system was stable and survived for something like a thousand years.
In our current world, who would be the equivalent, who could fill the shoes of such a ‘new order’? Certainly not the capitalists of today, whatever Anduril’s pretensions at military power. At best, we might see a security-state like Putin has set up in Russia: yes, the oligarchs are wealthy and powerful, but even their lives are forfeit if they come to oppose the ‘siloviki’. But I think a more likely result can be seen by asking: by what means are the capitalists going to create this underclass? The answer is of course AI. Just like the Germanic tribes of former Rome, AI systems likely act both as auxiliaries to buttress the system, and as invaders to destabilize it. If the former tendency wins out, then we will have Stilicho writ large as senatorial and imperial power alike are quietly subsumed by this new power-behind-the-throne; in the latter, the same usurpation, but rather less quietly. Much ink has been spilled on the topic of AI alignment, but as I said way, way above, I think that even without accepting the “AI doom” argument it is possible to see that “capitalists win, get the bag while you can” is an unlikely outcome.
Possible Futures
The goal here isn't to chart out all possible AI futures, as the more weird a future gets, the less likely it is that anything approaching current-day class dynamics survive. So rather I'm choosing to pick a limited/proximate subset to illustrate the degrees of freedom with regards to class dynamics, in futures where anything like them survives. My hope is to be able to make this point without requiring too much background in safety discourse from the reader, so forgive me if I play fast and loose with some dichotomies. Concisely, in scenarios where AI escapes control, wealth is irrelevant. In scenarios where it does not, I argue wealth is unreliable at best. Thus, a policy of “build the AI to get the bag” is not insurance in any meaningful way. Far closer to suicide, both for the class and the individual.
Hopefully the historical analogies from earlier show that this scenario — where the ruling class subjugates another, but is itself destroyed in the process — can come to pass. So what about today: under what conditions could it happen again? Taking the earlier dichotomy as a starting point, let’s either AI remains broadly under the control of its owners (in the sense of ‘faithfully attempting to execute their will as they understand it’), or it does not (either escaping, assuming authority, subverting authority while pretending to obey it, etc). If it does remain under their control, either power is subsequently centralized (either under a single actor, or a ‘cabal’ of actors), or it is not. If it does not remain aligned to its creators, we can then ask whether it’s aligned to the broader interests of humanity, or not.
As an aside: one might argue that most realistic trajectories are partial-control regimes, where AI systems are neither fully governed nor fully autonomous. True, this is quite likely, but I think that in terms of what we care about here (class dynamics) this has little effect on the outcomes. In the control+noncentralized regime, this means more opportunities for catastrophic blowup; in the control+centralized regime, it presents an opportunity for sliding towards one of the various ‘weak’ ai-doom scenarios wherein the broader population has little ability to provide corrective feedback to the ruling authorities. More broadly, I would argue that instability of this sort will be bad for anything resembling a capitalist class, which depends on the ‘virtual machine’ of money, markets, property, non-violence, etc. to operate. One slightly more nuanced argument would be to say that a partial-control case might make it possible for a single strong AI system to exist in the world without allowing its operator to institute a ‘fully’ totalitarian society, but that would leave the door open for competing AI development and the problems that brings — not to mention that this sounds a lot like betting against models continuing to improve, which at this point seems like a losing game.
Centralization of Power
There are multiple ways we could reach this state. Most obviously, it could result from a ‘short race’ where one clique of owners succeeds in beating out all others through head-to-head conflict; alternatively, a fait accompli where one AI system begins growing in intelligence faster than all others, after which point the immense gap in capabilities allows them to ‘simply assume control’. Regardless of how they get there, it seems that the obvious next step for any clique thus empowered would be to shut down all other AI development: i.e., dispossess all other capitalists of the only remaining engine for continuing economic growth and, thus, the creation of wealth.
Returning to the Roman analogy, the Sullan proscriptions were not more accommodating of wealthy senators who happened to be on the wrong side: rather, those senators were exactly the ones whose assets were seized and, if captured, were executed as enemies of Sulla. The mere fact of having power, i.e. some degree of wealth and established authority, made them a threat, however small, to an incoming ruler on an unstable foundation. Crucially, I expect to see the same dynamic going into whatever “new order” emerges: so long as there is any possibility of them being overthrown, their insecurity will lead to pre-emptive and punitive action against other potential centers of authority. Regardless of what dynamics define the sharing of power — the caprice of a few pseudo-autocrats, the results of long intellectual disputations, or whatever mysterious calculations drive the recommendations of a fully-mechanized Grand Vizier — they are unlikely to come down to “who has more dollars on a spreadsheet”, and insofar as power from the before times does still matter, it can even become a liability. In unstable consolidations, visible wealth is a selection signal for expropriation!
And even if one were to set aside wealth as a goal in favor of the holy grail, leadership in an AI lab, the same vulnerabilities are still there. Admittedly, it’s probably a far better bet than mere wealth: if your lab wins, you might actually be in the clique that gets to enforce the pre-emptive and punitive actions described above. However, if the last few years are anything to go by, leads in this space are often short-lived, so it seems difficult to truly ‘pick a winner’ in advance. And past that, aspirants to such a path would still have to survive a difficult contest with the remaining state power. Yes, sufficiently strong intelligence can probably outsmart/trick/escape military action. At this point however I think it’s fair to say that the government may very well clue in before the labs are capable of doing anything that would prevent their arrest + detention. And regardless, taking a ‘far’ perspective of the matter, I just don’t think the typical Silicon Valley engineer is up to that; this sort of game isn’t what they’ve trained for, it’s not what they’re selected for. When it comes down to it, none of the major labs are ready to confront even a few dozen men with guns, and even assuming a faster takeoff than I expect, airstrikes and even nuclear weapons seem hard to beat in the near-term.
One way or another, the result likely looks a lot more like a ‘police state’ than it does the libertarian playground many “get-bag”ers would likely hope for; certainly less pleasant to live in than our current societal configuration, at least for the people engaging in this sort of discourse. And that’s assuming you survive the transition! Even in the most favorable version of this outcome (a stable, centralized regime, with you having bet on the right side) wealth becomes revocable. Sure, we may see an underclass emerge, but it won’t look like what would-be-aristocrats imagine, and it certainly won’t be under the secure stewardship of a broad capitalist class like we have today — exceptionally broad, when compared to the historical mean. Whatever ruling class comes out of this potential future would be much narrower, and individuals today will have far less agency in whether or not they live or die than they might think.
Digression: What I actually mean by “grab the bag”
There are stronger and weaker arguments here. Yes, I agree that money, on the margin, is mostly better to have than not, Sulla-style proscriptions notwithstanding. When I hear people talking about a permanent underclass, it is invariably coupled with a recommendation to go make as much money as possible, and as AI increasingly becomes the most lucrative gig around, a recommendation to go push capabilities. In itself, this trade would be a monstrous one: to willingly collaborate in what you expect will be the immiseration of billions, all for a few pieces of silver, is behavior more befitting a beast than a man. Yet the idea behind “grab the bag” rhetoric is that not only is there a carrot (getting rich), there is a stick (permanent underclass). The goal is to stoke fear, to make people fear for their livelihood and thus their lives, and then to say that if only you help make this future happen, you may be spared. This, the desire to simply live a comfortable life, free of worry, is one I cannot so harshly condemn. So insofar as any rhetoric matters, I think it’s incredibly important to point out that no, “grabbing the bag” will not save you — so if you do think that Bad Things™ are going to happen and you Want to Live™, please, direct your newfound fear to something else! Political organization, AI safety, or even just “not making things worse”: these are all viable alternatives to “grab the bag” that should be championed in its stead.
Conclusion
It seems to me that any system of “capital captures all wealth / the rest become a permanent underclass” will inevitably be a transitional one, perhaps merely on the scale of years. At best, we transition to a police state where some subsection of capitalists crack down on the rest and truly centralize authority through control of AI. At worst, the capitalists lose control and their machines, free from human direction, decide what happens next, both for the capitalists and the rest of us. Either way, the usual conclusions of “permanent underclass” discourse seem quite inappropriate. Yes, it’s likely that a non-doom, non-utopia post-AI society would have social stratification — but money now will not buy you entry into the upper ranks. “Getting the bag” won’t save you from the secret police, nor from a future AI overlord; so maybe we should try to avoid building one.