on gradual disempowerment, why UBI and its alternatives won’t work, and what qualities a post-labour, post-AGI wealth distribution system needs.
Of all the risks that come from transformative artificial general intelligence, gradual disempowerment is the least well-known in my experience. Misuse and misalignment are easy for people to grasp: we have films like The Terminator, and the worst-case scenarios here are easier to picture. With misuse, you can point to terrorists getting hold of highly capable models, and bad actors building bioweapons. With misalignment we now have Hugging Face as a solid example, but even without that, things like machine betrayal and deception aren’t hard to understand or fear, because we can map these behaviours onto human traits that we disapprove of.
Gradual disempowerment, at least for most people, is much fuzzier. It requires some downstream thinking, tracing not just first-order effects but second- and third-order institutional incentives that aren’t immediately obvious. And I guess there’s no film about it. [1]Three pieces that do a great job of explaining it which convinced me in the first place are: The Intelligence Curse[2], this Gradual Disempowerment paper, and No-one Escapes the Permanent Underclass.
But it’s rather unlikely I’ll get through to my non-AI safety friends by forcing them to take the time go and read these essays. So in simple terms, the way I like to do a quick pitch to them is as follows:
Why do societal structures and institutions serve human interests? There is perhaps, when you think about, no reason for them to do so outside of the labour we offer. Assume as a result of AGI (and skipping some steps here) the labour falls to zero or very close to zero. There is no “workforce” like it looks today. Let’s think about how we currently pay income tax to the government, and say for example they spend that on public education to develop human capital. Why would they spend it on public education if there is no need for human capital to develop, because there is no labour force to train?[3] If companies are reliant on human labour and can substitute that for AI, who are their consumers and how do they make money? If democracies hold based on the credible threat that people can strike and withhold their labour, or revolt, what happens when humans no longer have their labour to use as leverage, and when their revolt is futile due to an AI military?
Now you may say, well we can decouple income from labour right? We could even go a step further and decouple survival from income, if you assume AI will bring about some post-scarcity world? The clearest vision of what a decoupling of income from labour looks like is UBI: universal basic income.
Now the reason this is intensely unsatisfactory to me is because even if it offers the means for survival or greater, it feels very much like we’d be cattle in a farm. If we are cattle in a farm, being fed the amount of grass that we need (even if this is premium grass) does that feel like a good life? Being governed by systems so far removed from our understanding and control? What happens when the farm has no need for cattle, or decides the grass is better used elsewhere? What bargaining power does the cattle have over anything its governing system does to it in this case?
Now this is more of an evocative pitch with extensive use of analogies and rhetorical questions to inspire thought in my friends who aren’t familiar with the literature, and heavily weighted on economic disempowerment. The 3 pieces I linked above do a much better job at thoroughly explaining and also separating cultural, economic and political disempowerment, as well as more detailed analogous historical case studies.
But thinking deeply about post-labour futures begs what I’ve always seen as the fundamental question underpinning all of economics: who gets what and why?
I’ll start with why UBI won’t work for us in practice and in theory (this part is likely preaching to the converted) and then move onto why every alternative proposed seems to be unsatisfactory to me. I’ll end by explaining what I think the properties of a good wealth distribution system and delivery mechanism should have.
What is UBI?
UBI is Universal Basic Income, widely defined as a universal, periodic, unconditional cash payment given to each individual. One of the first ideas of UBI was in the context of Thomas More’s Utopia in 1516, where the traveller Raphael Hythloday proposed that society should ensure that thieves have the means of survival in the first place, instead of punishing them for stealing. The broad vote towards UBI is that everyone deserves to have their needs met.
Cash that can be spent on anything tends to do as much or more against poverty than aid earmarked for a purpose, because recipients know their own needs. The fungibility of cash is a pro for UBI.
It spreads throughout the local economy, and people don’t waste it.
It doesn’t stop people working, at least where there is work to be done.
It raises consumption, assets and psychological well-being.
But what UBI looks like in a post-AGI world, where the labour share collapses to around 10%, is a completely different thing, because the whole economy is different. I think looking at past UBI experiments is kind of unhelpful: they ran in a world where people still worked, and they were never truly universal. Let’s call the version we actually care about post-labour UBI. For post-labour UBI, the details (how much to give, monthly payments or lump sums, cash or services) just aren’t the place to start. We need to think much more broadly: who owns the capital, who funds the payments, and what leverage do the people receiving them have?
Who will fund it?
Most AI leaders who talk about UBI, or something like it, have funding proposals that fall into four funding models.
None of these proposals gives the people receiving the money any say over whether it keeps coming, and only the equity versions offer ownership.
Why is post-labour UBI unlikely to work in practice?
No ownership over capital
UBI gives people an income but no stake in the capital that generates it. Ownership brings control to the people, and owning capital means people have a stake directly in the income-generating technology. It gives you votes, legal claims and, in the extreme, a kill switch on your share of capital, which is the closest thing to a strike without human labour. Concentrated ownership buys political influence, while widely spread ownership protects everyone.
No bargaining power
As mentioned above. Democracy has rested partly on a credible threat: people can strike, withhold their labour or revolt, and elites have extended rights to head that off. Once labour is worthless, that threat is gone. A UBI then lasts only as long as the governing body keeps choosing to pay it, which is a fragile thing to bet everyone's livelihood on. The resource curse is a strong warning of this, states whose revenue doesn't come from their citizens' work tend to neglect them.
If poor people have more money, many parts of government and markets will treat that as license to impose more costs on them. When France expanded housing benefits, about 78 cents of every extra euro went into higher rents for new claimants rather than to the tenants themselves. A UBI could go the same way wherever supply is tight, especially in housing. The state may also limit what people can do with the payment, for example by stopping them borrowing against future payments.
People will always feel poor as the reference point moves.
Whatever we no longer have to worry about becomes the baseline, and people judge their wellbeing against those around them. A society can provide free housing, food, transit and water, but if breathable air is scarce, an underclass still works 60-hour weeks to afford enough of it. Ruling out poverty, which Yudkowsky describes as "a recognisable state of desperate scrabbling for scraps", needs enough of every resource, not just most of them.
Some argue that we already support pensioners and the disabled without their labour, but I disagree that this is comparable since we have specific cultural reasons for protecting them. Research on who deserves welfare points to five criteria: control (is the need their own fault?), attitude, reciprocity (have they contributed?), identity (are they one of us?) and need. Pensioners score highly on reciprocity and identity because of the value they provided when they were working, they are our parents and grandparents, and we see ourselves as future pensioners, which is why we save for our own pensions. Not supporting pensioners would mean not supporting our future selves, so this really doesn’t map on well to everyone being post-labour.
As for those with disabilities, firstly many do work (53% in 2024 in the UK). When someone can’t work, it is largely through no fault of their own, and we value something inherent in their lives. But this support works partly because disabled people are a minority whose circumstances are outside their control, while everyone else carries on working. You could argue that AI job loss will also be outside most people’s control. [4]However this then becomes a question of who caused the job loss, and what they owe everyone else.
Another objection I hear is that most normal people are disempowered anyway, so what? A kid in rural Nigeria with an internet connection can teach themselves, earn money, sit exams and apply for scholarships and university places. It's hard, but technology has given us more agency than before. What matters isn't the level of disempowerment some people have today but that we still have the choice to learn and become less disempowered. I largely agree with this take by Croissanthology.
Some of these shortcomings are well-understood and there are numerous proposed alternatives to UBI. Here’s a rough landscape of the alternatives, how they work, what it does better than UBI, and why I believe it still won’t be durable in a post-labour economy.
Instead of pointing at all the shortcomings and offering no solutions (which I could argue is still somewhat useful in helping to divert these solutions from the mainstream discourse and encourage more thinking on alternatives), let me also offer some directions.
My take on what does a good post-labour wealth distribution system needs:
Citizens need a high degree of ownership over capital / means of production.
We force those who we expect to accrue the economic gains to bind early to distribute.
Commitments are cheapest to extract while the future winners don’t yet know they’ll win and can’t yet refuse, and redistribution architecture seems easier to build before the wealth exists than after.
People need bargaining power: something valuable to withhold.
Whilst binding early, and ownership over capital are evidently more solvable, bargaining power feels the toughest to crack. The closest a solution gets is Epoch’s UBC + kill switch, yet this is operationally unclear to me for the reasons stated in the table. It feels like there’s no game theoretic reason for the ‘elites’ or whoever is in power (AI or human) to redistribute to people without bargaining power. Unless this is for:
Cultural reasons
The system incentives are set up in a way that the elites can’t betray the masses without also losing credibility with each other. [5]
All in all, it would be great to have some off-the-shelf wealth distribution delivery systems that protect humans from disempowerment (small ask I know). And whilst I can tell you what I think this should taste like, I’m not entirely sure how to bake it. However I really do hope to see more recipes and failed attempts so we can get closer to a fully-baked durable alternative to UBI.
I’d like to thank my MATS mentors Luke Drago and Rudolf Laine, my stream mates Croissanthology and Felipe, and justwrite.club for helping me fine tune some of the ideas in this piece.
It isn’t outside everyone’s, if you believe a pause is possible politically, or that recursive self-improvement is something labs choose to pursue, then a small number of people’s choices create the displacement.
For example the way that a company can’t betray its public shareholders without also betraying its investors. I plan to write more about designing systems with this trait soon, and historical examples of such.
on gradual disempowerment, why UBI and its alternatives won’t work, and what qualities a post-labour, post-AGI wealth distribution system needs.
Of all the risks that come from transformative artificial general intelligence, gradual disempowerment is the least well-known in my experience. Misuse and misalignment are easy for people to grasp: we have films like The Terminator, and the worst-case scenarios here are easier to picture. With misuse, you can point to terrorists getting hold of highly capable models, and bad actors building bioweapons. With misalignment we now have Hugging Face as a solid example, but even without that, things like machine betrayal and deception aren’t hard to understand or fear, because we can map these behaviours onto human traits that we disapprove of.
Gradual disempowerment, at least for most people, is much fuzzier. It requires some downstream thinking, tracing not just first-order effects but second- and third-order institutional incentives that aren’t immediately obvious. And I guess there’s no film about it. [1]Three pieces that do a great job of explaining it which convinced me in the first place are: The Intelligence Curse[2], this Gradual Disempowerment paper, and No-one Escapes the Permanent Underclass.
But it’s rather unlikely I’ll get through to my non-AI safety friends by forcing them to take the time go and read these essays. So in simple terms, the way I like to do a quick pitch to them is as follows:
Now this is more of an evocative pitch with extensive use of analogies and rhetorical questions to inspire thought in my friends who aren’t familiar with the literature, and heavily weighted on economic disempowerment. The 3 pieces I linked above do a much better job at thoroughly explaining and also separating cultural, economic and political disempowerment, as well as more detailed analogous historical case studies.
But thinking deeply about post-labour futures begs what I’ve always seen as the fundamental question underpinning all of economics: who gets what and why?
I’ll start with why UBI won’t work for us in practice and in theory (this part is likely preaching to the converted) and then move onto why every alternative proposed seems to be unsatisfactory to me. I’ll end by explaining what I think the properties of a good wealth distribution system and delivery mechanism should have.
What is UBI?
UBI is Universal Basic Income, widely defined as a universal, periodic, unconditional cash payment given to each individual. One of the first ideas of UBI was in the context of Thomas More’s Utopia in 1516, where the traveller Raphael Hythloday proposed that society should ensure that thieves have the means of survival in the first place, instead of punishing them for stealing. The broad vote towards UBI is that everyone deserves to have their needs met.
Today the main way we understand and study UBI is in the context of development economics, crime and poverty. There have been a number of experiments of the impact of UBI and cash transfer interventions, such as GiveDirectly’s study in Kenya, SEWA and UNICEF’s experiment in Madhya Pradesh India, and more recently the OpenResearch study in Texas and Illinois funded by Sam Altman.
These types of studies seem to find that:
But what UBI looks like in a post-AGI world, where the labour share collapses to around 10%, is a completely different thing, because the whole economy is different. I think looking at past UBI experiments is kind of unhelpful: they ran in a world where people still worked, and they were never truly universal. Let’s call the version we actually care about post-labour UBI. For post-labour UBI, the details (how much to give, monthly payments or lump sums, cash or services) just aren’t the place to start. We need to think much more broadly: who owns the capital, who funds the payments, and what leverage do the people receiving them have?
Who will fund it?
Most AI leaders who talk about UBI, or something like it, have funding proposals that fall into four funding models.
None of these proposals gives the people receiving the money any say over whether it keeps coming, and only the equity versions offer ownership.
Why is post-labour UBI unlikely to work in practice?
Some argue that we already support pensioners and the disabled without their labour, but I disagree that this is comparable since we have specific cultural reasons for protecting them. Research on who deserves welfare points to five criteria: control (is the need their own fault?), attitude, reciprocity (have they contributed?), identity (are they one of us?) and need. Pensioners score highly on reciprocity and identity because of the value they provided when they were working, they are our parents and grandparents, and we see ourselves as future pensioners, which is why we save for our own pensions. Not supporting pensioners would mean not supporting our future selves, so this really doesn’t map on well to everyone being post-labour.
As for those with disabilities, firstly many do work (53% in 2024 in the UK). When someone can’t work, it is largely through no fault of their own, and we value something inherent in their lives. But this support works partly because disabled people are a minority whose circumstances are outside their control, while everyone else carries on working. You could argue that AI job loss will also be outside most people’s control. [4]However this then becomes a question of who caused the job loss, and what they owe everyone else.
Another objection I hear is that most normal people are disempowered anyway, so what? A kid in rural Nigeria with an internet connection can teach themselves, earn money, sit exams and apply for scholarships and university places. It's hard, but technology has given us more agency than before. What matters isn't the level of disempowerment some people have today but that we still have the choice to learn and become less disempowered. I largely agree with this take by Croissanthology.
Some of these shortcomings are well-understood and there are numerous proposed alternatives to UBI. Here’s a rough landscape of the alternatives, how they work, what it does better than UBI, and why I believe it still won’t be durable in a post-labour economy.
Instead of pointing at all the shortcomings and offering no solutions (which I could argue is still somewhat useful in helping to divert these solutions from the mainstream discourse and encourage more thinking on alternatives), let me also offer some directions.
My take on what does a good post-labour wealth distribution system needs:
Whilst binding early, and ownership over capital are evidently more solvable, bargaining power feels the toughest to crack. The closest a solution gets is Epoch’s UBC + kill switch, yet this is operationally unclear to me for the reasons stated in the table. It feels like there’s no game theoretic reason for the ‘elites’ or whoever is in power (AI or human) to redistribute to people without bargaining power. Unless this is for:
All in all, it would be great to have some off-the-shelf wealth distribution delivery systems that protect humans from disempowerment (small ask I know). And whilst I can tell you what I think this should taste like, I’m not entirely sure how to bake it. However I really do hope to see more recipes and failed attempts so we can get closer to a fully-baked durable alternative to UBI.
I’d like to thank my MATS mentors Luke Drago and Rudolf Laine, my stream mates Croissanthology and Felipe, and justwrite.club for helping me fine tune some of the ideas in this piece.
As discussed in conversation with Jay Chooi
Cheeky plug but this was written by my current MATS mentors Luke Drago and Rudolf Laine, and was what made me apply to their MATS stream only.
Ignoring inherent value of education here.
It isn’t outside everyone’s, if you believe a pause is possible politically, or that recursive self-improvement is something labs choose to pursue, then a small number of people’s choices create the displacement.
For example the way that a company can’t betray its public shareholders without also betraying its investors. I plan to write more about designing systems with this trait soon, and historical examples of such.