Human Empowerment in an AI Society

Alvin Ånestrand

Thanks to Benjamin Schmidt and Luke McNally for the discussions around this topic and their feedback on an early draft of this article!

Gradual disempowerment is one of the central failure modes of advanced AI. The idea, developed by Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, and David Duvenaud in their 2025 paper Gradual Disempowerment, is that humans need not be conquered by a rogue superintelligence to lose control of their own civilization. We can lose it slowly and quietly, handing off power over time, until the large-scale systems we depend on, such as the economy, the state, and culture, come to rely less and less on human participation and drift away from human interests.

Once this human participation gets displaced by more competitive machine alternatives, our institutions' incentives for growth will be untethered from a need to ensure human flourishing. Decision-makers at all levels will soon face pressures to reduce human involvement across labor markets, governance structures, cultural production, and even social interactions. Those who resist these pressures will eventually be displaced by those who do not.

I recommend reading Gradual Disempowerment before continuing, but this article can also be read on its own.

To understand this failure mode, this article looks at how disempowerment is already happening across our world each time people lose influence over their lives and the systems they participate in, whether economic, institutional, political, cultural, academic, or otherwise. Sometimes power concentrates among a few actors, as in an authoritarian regime, disempowering almost everyone else. Sometimes every actor is disempowered at once, with a system settling into a bad equilibrium that no participant wanted, like nations locked in an endless arms race. And sometimes it is human values, or human influence in general, that loses out: human-originated forces such as culture, ideologies, ideas, and creativity, no longer playing an active role in shaping the future^[1].

There are, in other words, many types of disempowerment. Whenever humans, or human-endorsed values, lose causal influence over decisions, infrastructure, and the future trajectory of the systems we live in, that is disempowerment. Individual loss of agency, people enforcing their wills upon others, humanity-level loss of steering ability, and cultural drift away from human origins are all examples of it, and all need to be addressed in the age of AI.

I focus throughout on disempowerment caused by systems; on how participants systematically lose power. Rather than dwelling on how individuals can be greedy and grab power from others, I think it is more constructive to ask how and why such people would succeed, which is a question of system design.

Even assuming that artificial superintelligence doesn’t simply outsmart humans and take over the planet (which would be an obvious and possibly sudden loss of empowerment)^[2], the future could be a world where many choose robot partners and friends that can be personally customized, where a few people (or AIs) control almost all capital and the planet’s resources, where states automate their economies and militaries and have little reason to invest in welfare, or where cultural trends are set by AI-created memes, art, and media.

We are already seeing a mild preview of all of this: people addicted to chatbots, the richest owning an enormous fraction of the world’s wealth, states deploying automated surveillance and military drones, recommendation algorithms and AI accounts shaping social media. Perhaps it isn’t that bad yet, not in most places at least, but the trend points toward disempowerment, and the problem isn’t well understood yet.

This article is my attempt to make sense of it, extending lessons from past and present societal problems to get a better idea of what may happen as powerful AIs are introduced and integrated. AI brings new challenges, but perhaps also new solutions. It has four parts, each answering one question:

Processes — what processes tend to drive disempowerment?
Underlying factors and amplifying conditions — under what conditions do those processes actually turn harmful?
Mechanisms against disempowerment — how do we improve those conditions, so the processes empower rather than disempower?
Decomposing disempowerment — what perspectives and frameworks help us reason about it more precisely?

Though this visualization may be more confusing than illuminating, I tried sketching out the core relationships between the processes, underlying factors and opposing mechanisms while working on this article:

The strongest relations between the Processes, Underlying Factors, and Opposing Mechanisms discussed in this article.

Bear in mind that these relations are largely based on my intuition, and that there are many more subtle relationships both between and within these columns. While built on a number of historical and recent examples, this framework hasn’t been carefully scrutinized by others or tested on its usefulness in analyzing previous societal transitions in detail.

If you find this interesting and want more analyzes on the future of AI, consider subscribing!

Processes

The processes below are not inherently bad. They can be engines of prosperity, and they only lead to disempowerment under the bad conditions described in the next section. What they have in common is that they trade empowerment for other values, such as efficiency or reliability.

Likewise, AI doesn’t necessarily amplify disempowerment within them, but it can when integrated in ways that, for instance, create dependency, manipulation, or opacity. I include a few examples of good integration here, but this section focuses on failure modes; constructive uses of AI are covered later in the Mechanisms Against Disempowerment section.

Competition

Fierce competition can force competitors to abandon their values in order to win, or just to stay in the race at all. They may fight over limited resources instead of coordinating to share them equitably. They can get locked into bad strategies that no one can unilaterally abandon. Some are disempowered when they lose; others are excluded from the competition entirely; and sometimes all competitors are disempowered at once, trapped in a system where no one is in control or can achieve what they actually want.

Competition is the force of evolution and natural selection. It is the force of capitalism. It is how leaders are selected in democracies, and how dictators take power in autocracies. And it is often the force pushing the other processes toward disempowerment.

The classic game-theoretic and decision-theoretic traps all belong here: the prisoner’s dilemma, the tragedy of the commons, Pareto-inefficient equilibria that no participant prefers but none can escape alone.

Competition can also be a force of progress and prosperity, as long as it is inclusive, fair, and its negative externalities and incentives are managed. But it is distressingly common that things go wrong and break.

The subsections below investigate a few ways in which competition brings disempowerment which I think deserve some additional attention.

How central competition is to disempowerment: Competition disempowers through destructive dynamics between participants, concentrating power among the winners, or trapping everyone in inadequate equilibria. It is probably the most central process of all, often the underlying force behind the others.

Main underlying factors: Irreversibility (bad equilibria), Control by Others (strong incentives to control others to win), Power Asymmetry and Inequality (winner-takes-all dynamics).

Too Many Competitors

When many people apply for the same job, grant, or other opportunity, the selection process leans on simple, reliable signals, such as years of education. Everyone then competes harder on those signals, perhaps studying for longer than the work itself requires. A larger applicant pool can help match the right people to the right roles, but the cost is that most participants end up comparatively disempowered, spending years chasing degrees and publications in prestigious journals instead of building competence in less legible ways.

The same thing happens when everyone competes for other people’s time and attention. Communication technology can, paradoxically, make it harder to reach people, build connections, find dating partners, or spread awareness of something important. A small threshold to taking initiative, even something as minor as the cost of a stamp, filters out the least interested competitors, and with a smaller inbox, each message gets more attention.

Here the disempowerment comes mainly from the sheer number of competitors, and it can be eased by keeping competition smaller and more local, or by adding a modest cost to participation.

With AI: AI can increase the number of competitors directly, with automated agents joining systems and crowding humans out. This is already visible on social platforms, where bots compete for attention—a mild preview of what may come later. It also amplifies competition between humans: people have started using AI to send applications, reach out, and share information. The number of competing agents grows even when the number of human competitors stays the same.

Fierce and Unforgiving Competition

When there is slack, many values can flourish, because participants can pursue them without being punished. When competition is fierce, actors are instead forced to give up their values for profit and power, or be outcompeted by those who do.

This is the familiar problem of unconstrained capitalism, but the same race to the bottom generalizes to competition among memes, cultural trends, religions, and ideologies. Globalization made many of these competitions fiercer at once, turning regional contests global.

The dynamics drive choices no individual competitor would prefer. Escaping requires better coordination, not better morals.

With AI: AI companies may be pushed to make their products addictive, and to make other unhealthy concessions, simply to stay competitive with one another. In the future, humans may feel compelled to defer to superhuman AI decision-making, handing over their power.

AI may also break the mechanisms that currently keep cooperation stable. It could defeat the verification methods that make agreements enforceable, or shift the offense–defense balance in domains like cybersecurity and biological weapons. On the other hand, it could just as easily become a powerful tool for cooperation — through privacy-preserving monitoring and hardware-assured compliance with agreements, for instance (discussed under Coordination in the Mechanisms Against Disempowerment section).

Losing and Exclusion

Disempowerment is also the natural lot of the losers, with power concentrating among the winners. Sometimes the problem is too little competition rather than too much. A monopolist buys up its rivals and startups, then charges whatever it likes; the market loses its variety and responsiveness, new entrants are shut out, and consumers can no longer shape it through their preferences. Patents and concentrated property ownership work similarly: when the only route to something valuable runs through a single actor, that actor can overprice it and block alternatives.

The cure is more empowered participants. Entering and staying in a competition should be easy, and the rules should be legible.

This is a core reason wide access to education matters. Incubators help startups get going. Antitrust laws curb anti-competitive conduct. Land value taxes discourage unproductive property hoarding. Problems like patent thickets and patent trolling might be discouraged through a Harberger tax, where a patent is taxed on its self-assessed value and anyone can buy it at that value at any time, forcing a sale.

With AI: The exclusion could become total. AI may compete with humans directly for resources, as workers and as consumers. In an economy of competing superintelligences, every less capable actor (humans included) could become largely irrelevant without necessarily being eliminated.

More broadly, human ideas, memes, and ideologies could be outcompeted by ones that originate and evolve inside AI or hybrid AI–human systems, and these can be arbitrarily bad for humans. Like a virus adapted to species A that occasionally infects species B, nothing limits how harmful they are to humans, because humans are not what they are adapting to.

Competition as Optimization

Efficient markets should optimize for customer needs, but customers aren’t always rational. We act impulsively, get addicted, and often engage more with what enrages us than with what informs or entertains us. We vote and decide based on what sounds good rather than what is good. Our behavior and revealed preferences are imperfect proxies for what we actually want, and competition built on those proxies can reliably make us worse off while doing exactly what it was designed to do.

Over time, markets keep finding new ways to exploit human nature, while countervailing forces like regulation and cultural adaptation try to keep up. Tasty but unhealthy foods spread, and diets and training regimes arise in response. New addictive drugs are developed, then largely confined to medicine. Engagement-maximizing platforms appear, then competitors offer more flexible or less addictive alternatives. The optimization process is broken and repaired, over and over.

With AI: Some people are already choosing relationships with AIs over relationships with humans. There is less risk; you can specify the looks, personality, and background you want, and even erase unwanted memories. AI embedded in robots may eventually be nicer and more attractive than people, and for many the offer may be too tempting to refuse. On a small scale this might be fine. But if it happens rapidly and at societal scale, will the countervailing forces have time to adapt? What does a society look like where humans have forgotten how to interact with other humans?

The same concern applies wherever competitive systems adapt to revealed preferences; markets, recommendation feeds, art, music. AI could become capable enough to supercharge that optimization in ways we cannot foresee or control.

Standardization

Standardization buys reliability, scale, and low cost, usually by reducing the room for human judgment while trading away flexibility, oversight and ability to intervene when something goes wrong.

David Krueger observes:

…society is increasingly “standardized” not only in terms of products, but also in terms of processes (e.g. restrictive customer service scripts or standard operating procedures) that have the benefit of being cheap, scalable, and reliable (often by eliminating “human error”, i.e. limiting human decision-making power and otherwise encouraging compliance). They also increasingly make more and more aspects of life subject to measurement and control via optimization of metrics, which necessarily fail to capture everything that matters.

The book Plurality by Audrey Tang, E. Glen Weyl, and their community, makes a related historical point (chapter 3-2): property, identity, and the vote all became standardized as people were abstracted into interchangeable individuals; atomic units that fit the system more neatly, at the cost of everything the abstraction left out.

But when a system keeps its flexibility, treats its participants well, and ensures humans stay involved to provide oversight and judgment, standardization can deliver quick and reliable solutions, and put a floor under how bad things get for any individual. Standardized measures of prosperity, progress, or even empowerment itself could be used by companies, communities, and states to combat disempowerment, as long as the measures are actually good.

How central standardization is to disempowerment: A core driver of disempowerment for individuals: the ones for whom the standardized solution does not work.

Main underlying factors: Irreversibility (standardized systems are often simpler, with large costs involved in replacing them and creating and maintaining more flexible procedures), Control by Others (standardization makes participants easier to manage and control), and Lack of Human Involvement (standardized procedures replace human judgment).

Standardized Optimization Targets

Standardized test scores let students demonstrate competence reliably, which is useful. But the measure misses a student’s full range of abilities, so schools and employers overlook talent that doesn’t show up on the test. Nation-states may optimize GDP rather than citizen well-being; companies follow the law rather than ethics. It’s Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure”.

Everywhere, actors chase measurable proxies for success, rather than what is actually good and desirable. In fact, the standardized optimization target often prevents participants from seeking anything else, punishing those who don’t optimize strongly enough for the measurable target.

With AI: AI is already used for classroom surveillance in China, narrowing what teachers and students can do:

The cameras record students’ head-up rates (as an indicator of attentiveness), the number of students seated in the front row, interaction patterns with the teacher, their facial expressions, as well as the teacher’s verbal tics, gestures, and whether their speech content triggers any “sensitive keywords.” In some cases, universities even mount a screen next to the classroom blackboard that displays these metrics as they update in real-time.

Beyond AI being used for measuring and optimizing standardized targets, the values embedded in the AIs themselves are a kind of standardization. Even if it becomes possible to install specific preferences in an AI, those preferences will likely be a lossy compression of what we (or, more realistically, what the developer) actually value. These values could get “stuck”, the AIs perpetually preserving them while acting in the world, leading to value lock-in.

Standardized Decision-Making

Standardized protocols speed things up where urgency matters, such as triage in hospitals, but often at the cost of handling rare cases badly. Done with care, standardization can keep a system running smoothly while preserving enough flexibility to fall back on human judgment when something unusual happens.

Done poorly, such decision making might discriminate against certain groups, like the widely used biased hiring algorithms, or force people to make bad decisions.

With AI: As technology improves, more complex decisions can be standardized. Huge, inflexible systems could be built and run by AI, implementing self-perpetuating protocols that may be very difficult to overturn, with humans unable to step in when things go wrong. Some domains that seem particularly likely to face these issues are the military, trading, and bureaucratic institutions.

Centralization

Centralization can take a few different forms. Perhaps single authority (whether a single ruler or majority decision) decides how things work, reducing flexibility and individual freedom; or a single system is put in place to organize a domain but imposes standardized protocols that don’t suit everyone. In return, centralization offers a single framework instead of a patchwork (important for effective regulation), improves coordination (as in the EU), and is often simpler, cheaper, and more scalable. It has many of the same advantages, and often overlaps, with Standardization. It is also a core way to prevent disasters, since centralized power often enables broader interventions and threat responses.

By default, things tend to concentrate over time across many domains: evolution (unicellular → colonial → simple multicellularity → complex multicellularity, and animals evolving to work in large groups), economics (artisan workshops → firms → joint-stock corporations → multinational conglomerates, with markets tending toward oligopoly and monopoly through economies of scale and network effects), and geopolitics (bands → tribes → chiefdoms → city-states → nation-states → empires and supranational blocs). The cost of centralization also falls over time: empires once collapsed under their own weight, but modern communication, information, and weapons technology made superpowers and global cooperation possible, along with monopolies and extreme concentrations of power.

How central centralization is to disempowerment: Centralization often causes disempowerment for many, while concentrating power among the few.

Main underlying factors: Irreversibility (strong incentives and supporting structures resist redistribution of power and decentralization), Power Asymmetry and Inequality (it often concentrates power), Control by Others (central authorities are often incentivized to control others to retain power), Principal-Agent Problems (central authorities have different interests from those they represent or rule over), and Lack of Human Involvement (decisions and actions are delegated to a central actor, reducing broader participation).

Reduced Flexibility and Individual Freedom

In most countries the educational plan is set by a central authority for the whole nation, reducing flexibility for individual regions, schools, and students. Everyone gets an education, but the system often removes the opportunity for students to truly thrive, with a pace and method that don’t suit everyone. Some countries soften this by offering alternatives such as homeschooling, grade skipping, single-subject acceleration, and Montessori-style learning.

Ideally, centralization and standardization put a floor under how bad things can get without putting a ceiling on how good they can be. Such floors—from federal aid, to free healthcare and education, to the basic security provided by police and military—keep individuals from being completely disempowered. Perhaps these could be provided without a central authority to administer them, but centralization seems to be the most common solution and mostly works reasonably well. Still, it doesn’t seem impossible to imagine alternative systems with more collective ownership and responsibility for such things, solving the same problems while allowing more flexibility and distributed empowerment.

With AI: A small number of companies own the most powerful AI systems and control how they behave and what they tell billions of users. This could enable persuasion and manipulation at unprecedented scale, perhaps subtle enough to avoid detection, while users can't easily switch to alternatives. AI also further enables power concentration, whether an AI gathers power for itself (as in takeover scenarios) or for humans. Authoritarian regimes could entrench themselves further, using automated oversight and military technology to keep their populations in check.

Less Involvement

Some have observed that there may be a decline in intermediary institutions mediating interactions between citizens and the state, including unions, churches, and local press^[3]. These give people a route to empowerment through community and a common voice, a route that may be eroding. I suspect the decline is partly a consequence of centralization: many of these institutions have been professionalized and delegated, so that membership now means paying a monthly fee rather than actually participating, getting to know the other members, and accomplishing things together. Communities are reduced to services^[4].

When power and resources concentrate among one or a few actors, the other participants naturally have less control over them, but they also tend to become less involved. They have less reason to understand how things work, and less reason to use whatever power and resources they have when the central actor does things for them. Even when such participants aren’t disempowered in principle, they are disempowered in practice, simply through the convenience of letting the center take over.

With AI: Deciding what values and objectives AIs ought to have has largely been delegated to a few frontier AI developers. These decisions could have long-lasting consequences as more of society is handed to AI, and the public should arguably have a say in how we want powerful AIs to behave. Ryan Greenblatt suggests mandatory interoperability to address this: “Pass regulation or create a norm that requires AI companies to support all the APIs and interfaces needed to customize their models and (attempt to) align them differently.” Done well, this would give users more options and let them pick AIs better aligned to them personally.

A separate issue is delegation to AI itself. Suppose there were a superintelligent system whose decisions almost always worked out well. The temptation to just let it decide would be strong; democratic countries might even vote to delegate decision-making to a centralized AI authority, because it simply works better. Perhaps we get an AI that excels at legal judgment, and countries start routing cases through a central AI judge. Then a central AI administrator for the economy? For banks? For international relations? But this is a dangerous path: humans become less involved, lose their understanding and skills, and grow dependent on the central AI.

Broken Incentives

Centralized decision-making often comes with broken incentives for the decision-maker. A central authority doesn’t necessarily benefit from making good decisions, and those at the top may not benefit from empowering those at the bottom^[5]. Even democracies suffer from this: a majority can set rules that serve them at a minority’s expense. Existing homeowners, for example, have a direct interest in their property values rising and therefore in new housing not being built, plausibly one contributor to the housing crisis. A central authority may care mostly about the groups large enough to threaten its power or swing an election.

With AI: Even if the values embedded in AIs were somehow decided democratically (already far better than today, where there is no such participation and alignment isn't solved in the first place), those values might still fail to represent many or most people alive, let alone past and future ones. Even authoritarian regimes today can't fully afford to antagonize their people or their military, which might revolt. But if the military is automated with robots and autonomous weapons, and humans can be replaced with government-controlled AI, that constraint disappears: the government would no longer need to consider the people, who could no longer protest. And if AIs are ever granted rights such as the vote, enough of them could impose rules on humans, even if the decision is in some sense “fair”.

Automation

Automation closely resembles Standardization: automated processes stop relying on humans and human judgment, raising efficiency and reducing error at the cost of flexibility, handling of special cases, and the ability to respond when things break. Skills for doing things manually atrophy, producing dependence on the automated solution, which is often opaque to those who depend on it.

With AI: Automating work tasks could displace a great many people. Historically this has mostly caused temporary disempowerment, with people being able to eventually find other work. But AI could cause a far larger disruption, advancing and spreading at unprecedented pace, until the only tasks left for humans are those required by law or strongly preferred to be human (for example, human-made art may keep its value for its origins, and many will still want human therapists).

A separate issue is that investment in AI could yield higher returns than investment in people. This is the intelligence curse: governments have less reason to ensure the welfare of their populations if they can automate humans away and suppress unrest with AI-powered surveillance and militaries (see the broken-incentives discussion under Centralization above). More abstractly, automation makes humans less influential in general—a disempowerment not just of the people who were replaced, but of human influence, from creativity to ideology to judgment to impulsiveness.

How central automation is to disempowerment: Automation takes humans out of the process, disempowering those who are replaced and reducing human oversight, flexibility, and control.

Main underlying factors: Irreversibility (deskilling produces dependence), Principal-Agent Problems (automatic processes or agents fail to embody all of the principal's interests), and Lack of Human Involvement (automatic systems replace human judgment).

Upscaling

Some systems grow until they exceed human comprehension. Law and bureaucracy that is never trimmed becomes labyrinthine; markets grow so large and interwoven that attempts to regulate them backfire.

When systems scale up, they often become illegible to their own participants. The system becomes an Eldritch being that may or may not be benevolent, one humans cannot oversee, where attempts at change fail or cause more harm than good.

Scaling up coordination, on the other hand, may be exactly what’s needed to fight disempowerment: we can scale up collective sense-making and decision-making too, increasing collective power without necessarily reducing individual power. The problem isn’t scale as such, but scale that makes a system illegible or otherwise inhibits human oversight and control.

With AI: AI lets systems expand far past anything we could oversee. Merely by participating in existing social, economic, or academic systems, AIs can complicate them by multiplying the number of participants (see the Too Many Competitors subsection above). As communication and information technology has let systems expand, AI will too, as the availability of intelligence stops being a bottleneck. Systems would no longer be constrained by the need to stay legible to humans. On the other hand, AI could also help scale up collective sense-making and coordination (see Coordination under Mechanisms Against Disempowerment).

How central upscaling is to disempowerment: Upscaling often makes it more difficult to handle disempowerment caused by other processes, reducing the legibility, oversight and control needed to correct course.

Main underlying factors: Irreversibility (systems grows complex or powerful enough to resist change), Illegibility (the system grows too large and complex for participants to interact with it effectively).

Speedup

Speedup is the sibling of Upscaling: systems accelerate beyond the human capacity for meaningful oversight, becoming illegible and hard to change.

When change is slow, a rising destructive force tends to be met by a balancing one. A predator that hunts its prey to extinction follows it into extinction, but in a slow, dynamic ecosystem the prey has time to adapt. One of the balancing forces in human society seems to be benevolence: when things aren’t too fast or chaotic, people tend to push back against destructive forms of profit- and power-seeking. Speed makes it difficult to retain such balance.

With AI: For any task an AI can do reliably, it's usually many times faster than humans. This has held across domains, from chess to coding to writing short stories. This speed inhibits human oversight.

As AI speeds up the processes inside systems, or acts as a participant itself, things could happen at unprecedented speed. Just as algorithmic trading enabled flash crashes, AI decision-making in military or political contexts could enable flash escalations, as in the Future of Life Institute's Artificial Escalation video, with balancing forces, de-escalation, and coordination unable to keep up. AI research itself faces the same risk: recursive self-improvement could make the field move so fast that there's no time to respond when something goes wrong.

As with Upscaling, though, AI could also support constructive efforts, like speeding up research into hardware assurance to enable trustworthy and enforceable treaties on AI development and deployment. Speedup is mostly a problem when it causes illegibility and erodes oversight.

How central speedup is to disempowerment: This is perhaps the single most important factor in whether we can respond to disempowerment at all: these problems may simply emerge faster than we can react.

Main underlying factors: Illegibility (speed outpaces our ability to understand what is happening in time to change course).

Local Optimization (Greedy Algorithms)

Many systems are built in ways that reward shortsightedness. Actors do what is easiest now rather than what is best overall, and systems settle into local optima instead of global ones.

Democracies often reward politicians for delivering temporary improvements that boost reelection chances; cash or tax benefits may be popular but can simply push prices up. Deep problems like the housing or fertility crisis may require costly, controversial reforms that are risky for whoever attempts them, which discourages serious solutions. Shortsighted fixes fail to empower people in the long run. It’s the problem of policy makers responding to crises after they occur instead of preparing for them. It’s the problem of people not noticing or reacting to gradual catastrophes, aka the “boiling frog syndrome”.

Illustration by ChatGPT, Ghibli style. I hope readers appreciate the irony of using AI art in this article.

If we optimize only for the people alive today, we disempower the people of the future. Climate change is the key example, gradual enough that today’s decision-makers don’t expect to bear its full cost. That gradualness means the decision-maker isn’t the one who benefits from a solution (see Moloch’s Toolbox). Investment in education and welfare empowers a population, but it’s a long-term investment, which is probably a core reason non-democratic countries have historically struggled with it. Where natural resources are abundant, governments may be tempted to invest in those resources rather than their people, profiting in the short term while stunting long-term growth, a problem known as the resource curse.

With AI: Each decision to hand power to an AI system may be locally rational, even as the disempowerment accumulates over time. Since this process is likely spread out over time and across many domains, people may fail to notice and respond in time.

How central local optimization is to disempowerment: This is perhaps the process that best captures the gradual in gradual disempowerment: each individual step can be locally optimal while steadily reducing our ability to reverse course.

Main underlying factors: Irreversibility (problems are often easier to solve if nipped in the bud, but shortsightedness prevents this), Principal-Agent Problems (decisions are delegated to actors rewarded for shortsightedness).

Underlying Factors and Amplifying Conditions

The processes above don’t always end badly. Whether they disempower depends on a set of background factors and conditions. This section covers the factors that cut across several processes; issues tied to a single process (like the game-theoretic traps under Competition) were brought up alongside it.

Irreversibility

The processes causing disempowerment are often difficult to reverse. We become dependent on new solutions, forget how things were done before, and lose our ability to change course. We become stuck in suboptimal equilibria, unable to move to better systems. When empowerment is lost, this further reduces our ability to change the system, turning disempowerment processes into negative spirals, until the threshold of no return is crossed.

This takes countless forms. Infrastructure becomes load-bearing as systems can no longer function without it (like dependence on digital systems leaves us vulnerable to blackouts and cyberattacks once manual backups are no longer maintained). Old ways of doing things are lost to deskilling. There are network effects, path dependence, vendor lock-in, and political entrenchment. Systems grow harder to change as they become illegible, too fast, too large, or too complex for oversight. And the capacity to respond to threats (to coordinate, and to devise and implement solutions) can itself be lost. Cultures, ideologies, norms, and even values shift over time, often in ways that resist reversal, for good or ill.

Control by Others

There are strong incentives to manipulate and control other people, and being controlled is itself a core form of individual disempowerment. Two of the most worrying incentives are profit and power. For profit, people sell addictive products like drugs and betting platforms; recommendation algorithms prey on our attention and money; and marketing and scams exploit human weaknesses, from superstimuli to gullibility. For power, there’s persuasion, propaganda, indoctrination, and surveillance, alongside blunter tools like extortion and blackmail.

Power Asymmetry and Inequality

When power and resources are concentrated among the few, or otherwise unequally distributed, the less fortunate end up disempowered: people lacking wealth, influence, or the ability to coordinate can’t change their own circumstances. The relevant inequalities range from tangible things like income, to less tangible things like status and connections. Beyond any intrinsic value, equality tends to be instrumentally healthy: lifting people from poverty can expand the labor supply, diverse teams can foster creativity and innovation, and equality can enable better coordination since negotiation is often easier on equal ground.

Wealth inequality is perhaps the most salient example, and worth expanding a bit on:

Wealth Inequality

The richest 1% hold on around 36.4% of global wealth, according to the World Inequality Database. This wealth share has been broadly stable since around 1980.

However, the inequality between the extremely wealthy, top 0.001% and the rest of the world is steadily increasing. The top 0.001% owns about three times more wealth than the entire bottom half of humanity:

Source: World Inequality Report 2026

AI automation threatens to widen inequality further by shifting income from labour toward capital, amplifying a trend that appears to have started decades ago:

Source: Global Macroeconomic Imbalances after the Crisis: From the Great Moderation to Secular Stagnation

This process moves economic resources away from laborers to capital owners. Also see the discussion on Automation for how this may cause disempowerment.

Note that this transition may be very gradual, with many economic sectors potentially being quite resistant to AI automation. While labor share of GDP may be an important aspect of AI-associated disempowerment, the effects will likely be very unevenly distributed.

Power Asymmetry Spreads Between Domains

One type of power asymmetry or inequality tends to spread into others. Economic inequality becomes legal inequality where court outcomes depend on whether you can afford skilled lawyers, and political inequality where money buys lobbying and election campaigns.

In the same way, access to AI might become the determining factor for most forms of societal inequality. If you have the best AI, you can use it to accumulate capital, improve your decisions, and outmaneuver rivals. Intelligence looks like the ultimate resource, and if we don’t ensure equality in this domain, it will open a vast chasm between those who have it and those who don’t. That chasm could just as easily run between superintelligent AI and the rest of humanity.

Inequality and Societal Transitions

Power asymmetries seem to underlie key societal transitions. The industrial revolution redistributed capital across social classes, creating pressure toward power equalization that helped enable the move to democracy and liberalism (though this is a gross oversimplification of everything involved).

With the rise of AI, automation may instead enable further concentration of capital among the few. The incentive for states to invest in their people weakens (the intelligence curse), as AI replaces human labor as the main source of state income and the pressure toward concentration grows, which could push toward oligarchy or autocracy.

Principal-Agent Problems

Delegating work, decisions, and power risks disempowerment along several paths: the agent (the one delegated to) may have different interests from the principal (the one delegating); the principal becomes less involved in the delegated task, leading to deskilling and weaker oversight; there may be failures of communication or understanding between them; and the principal may lack the means to punish, change or replace the agent when it doesn’t act appropriately on the principal’s behalf. There’s also no comprehensive theory of empowerment to tell us how an agent could best empower its principals. The science of agency, power, and values strikes me as underdeveloped.

With AI there’s the added problem that it often can’t explain its decisions and reasoning, and we don’t yet understand its internal workings, which makes delegation to AI inherently unreliable, at least until interpretability research matures considerably.

Lack of Human Involvement

Specific humans can be disempowered through displacement or restriction, but when humans in general are removed from a system’s core functions, it produces a broader disempowerment of human influence. Human participation (through oversight, judgment, decisions, action, communication) tends to keep some moral consideration present in a system, improves its alignment with the people in it, and provides flexibility, since humans can step in when automated or standardized procedures fail.

Illegibility

Even where a system allows for human involvement in its core functions, the humans may be unable to provide meaningful oversight once it grows too large, fast, or complex.

Mechanisms Against Disempowerment

Having looked at which processes lead to disempowerment, and when and why, this section turns to the opposing forces, the mechanisms toward empowerment.

This section is more exploratory than prescriptive. There are many approaches I think would help, but I don’t claim to know how they’d be implemented in practice, what the exact steps would look like, or how to convince people they’re good ideas. The aim is to map the space of possible solutions, categorize them, and figure out which part of the problem each one addresses.

Some of these, particularly Utility Maximization, Coordination, and Slack, are inspired by Scott Alexander’s Meditations on Moloch. Another core inspiration is Plurality, with its many ideas for designing systems that embrace diversity and keep participants empowered.

The mechanisms overlap and reinforce one another. Coordination is often the method used to redesign systems toward Utility Maximization and Balance; Slack and Understanding help those efforts succeed. Just as disempowerment processes are self-reinforcing, the mechanisms against them feed into each other too, enabling positive spirals rather than negative ones.

Utility Maximization

Processes like Competition and Local Optimization can be highly constructive when aimed in the right direction. When markets work well, they deliver products that fill people’s needs at low cost. In systems where local incentives are aligned with long-term prosperity, shortsightedness stops being an issue. This is the power of well-designed incentive structures and metrics that measure what people actually care about.

Some levers here are transparency (for efficient markets), baseline alternatives (basic, perhaps state-provided versions of a service to fall back on when other options turn exploitative), and collective decision-making (which accounts for diverse interests better than delegation does). It’s closely tied to the Balance mechanism below; for instance, when leaders can be replaced by those they lead, it holds them accountable and incentivizes better decisions (though this doesn’t always work perfectly).

An interesting approach to utility maximization in the age of AI is full-stack alignment: “the concurrent alignment of AI systems and the institutions that shape them with what people value.” It proposes to represent values using thick models of value (TMV), described as “a broad class of structured approaches to modeling values and norms” that aim for greater robustness against distortion, better treatment of collective values, and better generalization than alternatives like utility functions, preference orderings, or natural-language descriptions.

Thick models of value applied to a recommender system, from the Full-Stack Alignment article

Institutions and other systems aren’t always well described as strict hierarchies as in the figure above, though; the more general idea is that values should be preserved and transferred between interacting system components and across intersecting networks of individuals and groups, while taking the broader context into account (negative externalities, effects on culture and norms) and optimizing for collective values as well as individual needs (such as community trust).

While I call this mechanism “Utility Maximization”, I don't mean that there's some single, clearly specified utility to be maximized. We don't even always know what we want. I think it's more constructive to ask how system participants can determine the shape and direction of the system themselves. That seems to be a core feature of full-stack alignment, which tries to retain as much as possible of the complex values and interests of participants rather than prescribe how a “good” system should work.

Utility maximization may sound good, but can we shape our systems like this, given all the problems discussed in previous sections? I think some optimism is warranted, because utility maximization is often just better. With empowered participants, systems tend to run more smoothly. A core reason democracy spread could be that it was more effective than the alternatives, with states that adopted it growing stronger than those that didn't. By the same logic there may be still better democratic governance structures, ones that empower people even more, that would make today's democracies look brutish by comparison.

Another example is the steward-ownership company structure, where control of the company is not for sale. Owners act as “stewards,” steering the company but unable to extract profits at will or sell it for personal gain; investors can be compensated but don’t get direct control. Steward-owned companies are reportedly more resilient in crises, longer-lived, more sustainable and innovative, and have happier employees (though I haven’t verified this through independent sources yet).

With AI: The full-stack alignment article suggests several ways AI could support constructive incentive structures and the transmission of values through a system: from helping people clarify and pursue their own values, to enabling new and more robust measures of human flourishing, to AI representatives that negotiate on behalf of people and groups at AI speed, to “meaning-promoting market intermediaries”: ”Instead of consumers paying directly for services based on simple, often misaligned proxies (like subscriptions or engagement), the intermediary would pay suppliers based on their measured contribution to the flourishing of their customers as expressed in their own values”.

A related proposal is Civic AI: “artificial intelligence that answers to the people it affects. Instead of one powerful system built to govern everyone, the idea is to build many small ones that a community can own, inspect, correct, and switch off.” These local stewards, which the framework calls Kamis, are meant to help “neighbourhoods, schools, unions, faith groups, cities, and diasporas do what collective self-government has always promised but rarely delivered at scale: listen across difference, deliberate in the open, remember faithfully, and act together. No central model owns them; communities govern, inspect, contest, and can shut them down.” The structure would enable collective decision-making at larger scales while keeping participants well represented.

Main factors this addresses: Control by Others (people are incentivized, and procedures are designed, to strengthen participants rather than exploit them) and Principal-Agent Problems (agents and principals are broadly aligned through systems built to handle diverse interests).

Coordination

Disempowerment makes coordination harder over time. Authorities entrench themselves; systems become too large, fast, opaque or complex for human oversight; and humans lose the ability to come up with or implement better solutions. Worse, most people may not notice that anything is wrong, be distracted by cool AI toys, and placated by seemingly good welfare policies that may point in the right direction but fail to prevent disempowerment long-term (such as universal basic income policy implemented without any other guardrails ensuring humans stay relevant in a mostly non-human economy).

But coordination is powerful. Perhaps new advancement in coordination technology could make it strong enough to even replace competitive pressures in driving societal changes. There would be collective, deliberate, long-horizon steering. It would be what Owen Cotton-Barratt and Raymond Douglas call a choice transition. What follows is a discussion on how to reduce coordination friction (a.k.a. transaction costs), improve collective sense- and decision-making, and how the social contract may change.

Main factors this addresses: Irreversibility (coordinated efforts are stronger than individual initiatives, enabling more ambitious solutions), Power Asymmetry and Inequality (many weaker participants can join together for leverage), Principal-Agent Problems (good coordination strategies often reduce the need to delegate tasks and decisions to a central authority), Lack of Human Involvement (by nature, coordination requires participation)

Reducing Transaction Costs

Some agreements would make at least one person or group better off while making no one worse off (so-called Pareto-improving agreements), yet never get implemented. There are states of the world strictly better than this one that we simply can’t reach, because the friction, the transaction cost, is too high.

Here I’ll defer to an excellent article by Nora Ammann, Gradual Paths to Collective Flourishing. Ammann proposes new infrastructure, largely enabled by AI, to drive down transaction costs:

Information infrastructure that makes it cheaper to figure out what’s true and share it appropriately
For example: scalable world-modeling infrastructure like shared ontologies and ‘living’ knowledge graphs, privacy-preserving computation and mechanisms for aggregating distributed information like prediction markets, reputation systems and sensor networks.
Deliberation infrastructure that helps individuals and groups understand what they actually want.
For example: preference elicitation and structured reflection aids, infrastructure for collective sense-making, deliberation, and imagination.
Bargaining infrastructure that makes it easier to reach, specify and execute complex multi-party agreements between heterogeneous actors, including mechanisms that are robust to strategic manipulation.
For example: AIs capable of using or generating strategy-proof protocols or programmable cryptography, AI delegates with verifiable constraints and nuanced principal-specified affordances
Trust/Assurance infrastructure that ground digital claims in physical reality, thereby driving down monitoring and enforcement costs.
For example: secure hardware, tamper-evident sensors, verifiable computation, guaranteeable actuators.
Cutting across all of these are scalable oversight solutions: infrastructure that allows humans to gain justified confidence in AI outputs — be that in science, engineering, or decision-making — even as AI systems handle more of the work. Even if AIs are essentially aligned, blind trust is not robust. AI systems still can make mistakes, misunderstand tasks (including because the instructions may be genuinely ambiguous), be subject to sabotage, or similar. Solving this unlocks AI-AI coordination (agents can prove things to each other) and AI-human coordination (humans can maintain oversight even as AI capabilities grow). Without it, we either don’t use AI (and fall behind) or use it without adequate assurance (and introduce new risks).
These layers together form a shared trust protocol: a stack where each layer enables the others, and the whole becomes a foundation for coordination at scale. Trust infrastructure grounds information infrastructure; accurate world models support deliberation; clear preferences enable efficient bargaining; enforceable agreements close the loop. And finally, surplus from cooperation funds further investment in the stack.

Better Democratic Processes

Apart from reducing transaction costs, perhaps we can advance the underlying societal structures themselves. Democracy is, after all, also a technology, and one that hasn’t seen much development in a while.

Perhaps decision-making could be more local, transparent, and collective, as in the Civic AI framework. AI could be used to aggregate views more effectively, and negotiate on behalf of those that are absent from decision-making processes. Alternative voting schemes like quadratic voting let voters express the strength of their preferences, so minority issues can be addressed when the minority cares about them intensely while the majority doesn’t.

A New Social Contract

The social contract defines the relationship between individuals and authority, whether a state, or the decision by the majority. People surrender certain freedoms in exchange for protection and social order. It is the system of rights, authority, freedoms, and responsibilities, the rules by which individuals and groups operate as parts of a larger whole.

That contract can shift significantly with new technology. Some technology lets us surrender fewer freedoms while keeping the benefits of order and rights, like privacy-preserving identity verification. Other technology, like AI-powered mass surveillance, can push the contract toward authoritarianism. AI could break the current contract entirely, producing the kinds of freedom-destroying scenarios described earlier. Or it could improve it, lowering coordination costs to enable new forms of agreement, or supporting better democratic processes.

My point is that we shouldn’t focus only on collective decision-making like democracy, or on welfare policies to cushion the AI transition. We need to discuss the whole social contract: how it should work and how it can be enforced; rights, roles, and responsibilities; from small communities to international alliances; within and across law, politics, markets, education, and other societal domains.

There is a severe lack of attempts at describing a well-functioning world with AGI or superintelligence. Even the AI 2027 scenario mostly leaves the long-term societal changes to the imagination of the reader.

Balance

So far, something has always stopped any single entity from gaining total control over Earth. That wasn’t obvious, given enormous militaries, huge monopolies, and the ability to coordinate globally. Concentration of power has grown over time (from small groups to states to empires to superpowers), yet forces keeping competition alive remain. Maybe we can learn from them.

When one nation has a stronger military than another but the cost of attacking is too high, a balance of power persists despite the asymmetry. Nuclear weapons have their own term for their balance: Mutually Assured Destruction (MAD).

Balance is not opposed to asymmetry and inequality. It can preserve empowerment despite them, though leveling the playing field helps. Reducing income differences through deliberate political choice has historically been very hard; redistribution methods like Universal Basic Income (UBI), wealth floors, public ownership of compute or Universal Basic Compute (UBC), and citizen dividends might marginally support empowerment but probably won't much improve equality or symmetry without extreme effort (and, as the resource curse warns, such efforts often fail if there's little incentive to invest so much in the public). But balance might be easier to achieve than full equality, alleviating the disempowerment that asymmetry causes.

Many levers belong here: separation of powers for balance between state powers, rights like freedom of speech, whistleblower protections, antitrust laws.

Methods toward balance fall broadly into two kinds, which I will call vertical and horizontal balance. Vertical balance operates between actors at different levels of a hierarchy (authority levels, individuals and collectives, employees and employers). It concerns how lower levels stay empowered despite surrendering some freedom to higher ones: how we prevent abusive workplaces, ensure minority interests are considered, and build a healthy social contract^[6]. Here belong consumer rights and the options of voice (democratic participation, protest, journalism, whistleblowing) and exit (antitrust regulation, the ability to change employer or migrate), as explored by Albert O. Hirschman.

Horizontal balance operates between actors in similar roles, and concerns how competition and conflicting interests are managed without falling into the traps under Competition. Sometimes a higher authority is needed to provide order, which risks disempowerment through Centralization but can be checked by vertical balance. Here belong social norms, international treaties, MAD, and assurance methods that hold parties to agreements regardless of how unequal they are.

There can of course also be balance between completely different types of actors that don’t fit into these broad categories, but that would be more case-specific and harder to discuss in general terms.

Main factors this addresses: Irreversibility (balance keeps some actors empowered enough to respond when things go wrong), Control by Others (people are empowered to resist manipulation), Power Asymmetry and Inequality (it preserves empowerment despite asymmetry), Principal-Agent Problems (neither party can exploit the other), and Lack of Human Involvement (balance is usually achieved through mutual checks and supervision).

Benevolence

A benevolent, all-powerful leader, freed from competitive pressure and bad incentives, could in principle solve every problem in this article, empowering everyone else like a caring parent while keeping ultimate authority. But such paths are dangerous, risking dystopic authoritarian outcomes. Securing accountability and balance without reintroducing the very incentives this approach was meant to avoid is hard; even democracies haven’t managed it (yet). And how would we know whether a would-be leader is truly benevolent?

Still, benevolence is central to preventing disempowerment. It’s a force so powerful that it often faces down the terrible forces of destructive incentives and vicious competition, and comes out on top. As Scott Alexander puts it:

The Universe is a dark and foreboding place, suspended between alien deities. Cthulhu, Gnon, Moloch, call them what you will.
Somewhere in this darkness is another god. He has also had many names. In the Kushiel books, his name was Elua. He is the god of flowers and free love and all soft and fragile things. Of art and science and philosophy and love. Of niceness, community, and civilization. He is a god of humans.
The other gods sit on their dark thrones and think “Ha ha, a god who doesn’t even control any hell-monsters or command his worshippers to become killing machines. What a weakling! This is going to be so easy!”
But somehow Elua is still here. No one knows exactly how. And the gods who oppose Him tend to find Themselves meeting with a surprising number of unfortunate accidents.
There are many gods, but this one is ours.

Most people are actually nice to each other. Things go wrong mostly because of structural problems, not because of malice. Things go wrong when the offer to sacrifice everything we love for power, or efficiency, or a temporary edge stays open. And benevolence, like the other mechanisms, can be strengthened. It’s a property of culture, norms, ideologies, and religions as much as a trait of individuals, and it belongs to institutions and system structures as much as to interactions between people. Approaches like thick models of value, and pluralistic designs with more collective sense- and decision-making, might let benevolence play a larger role in our institutions.

Main Underlying Factors this addresses: Control by Others (actors with the power to control others choose not to), Power Asymmetry and Inequality (the powerful choose not to exploit the weak, and honor agreements and healthy norms), and Principal-Agent Problems (agents genuinely try to empower their principals).

Understanding

Disempowerment can often be avoided fairly easily, as long as people understand what’s happening. Exploitation is much easier when the exploited don’t know about it and accept their situation as simply the way things are. People who recognize common manipulation tactics (in politics, advertising, gaslighting) are better prepared to resist them, and systems that are legible and transparent let us think carefully about what we can and should do.

The relevant disciplines, including but not limited to sociology, political economy, decision theory, philosophy, psychology, forecasting, and epistemics, strike me as underdeveloped and unready for the challenge of fighting disempowerment. We need better theories of empowerment and agency.

AI cuts both ways here: a tool for misinformation through deepfakes and automated propaganda, or a tool to help people research and understand their own circumstances. Ryan Greenblatt suggests giving every human an AI representative and advisor:

If every human had a competitive and aligned AI representative which gave them advice on how to advance their interests as well as just directly pursuing their interests based on their direction (and this happened early before people were disempowered), this would resolve most of these concerns.

This seems like it might help at the margin, but, like UBI or UBC, it hands people a basic resource without addressing the underlying inequality and unhealthy competitive dynamics.

AI could also help accelerate progress in the disciplines above. It would be very tragic if AI caused irreversible disempowerment before it was capable enough to help us solve it.

Main factors this addresses: Control by Others (those who understand how others might control them can resist it), Lack of Human Involvement (it's easier to take part in processes you understand), Illegibility (improving understanding improves legibility), and, more broadly, every other factor (a better understanding of disempowerment helps against all of them).

Slack

When basic necessities are hard to come by, competition turns fierce, and when losing means losing everything, people take drastic measures to avoid it. Slack (excess capacity, spare resources, redundancy) is therefore protective. As Scott Alexander puts it: “As long as resources aren’t scarce enough to lock us in a war of all against all, we can do silly non-optimal things – like art and music and philosophy and love – and not be outcompeted by merciless killing machines most of the time.”

Slack isn’t only about physical resources. It’s also about time and attention. As systems and institutions grow more complex and more things compete for our attention, our mental resources grow scarcer. There’s an inflation in personal effort, where the same effort buys less than it used to, across domain after domain.

Slack is what gives the other mechanisms, especially Balance and Utility Maximization, room to operate. Without it, there’s no margin in which to coordinate, deliberate, or resist.

Main factors this addresses: Control by Others (less reason to control others when there's enough to go around) and Power Asymmetry and Inequality (even under inequality, most participants can stay largely empowered when there's enough for everyone).

Decomposing Disempowerment

The original Gradual Disempowerment paper focuses on three systems (economy, culture, and state) while noting the argument extends to others, like research or law. But there are several useful ways to decompose disempowerment, allowing for a more nuanced discussion. Each offers a different angle on the forces involved.

By societal system: economy, culture, state, (science, law).
This is the division explored in the original gradual disempowerment article. It examines how AI may cause disempowerment in each of these systems, as well as how disempowerment in one system can spread to and reinforce disempowerment in other systems. When pointing to issues in one system (like inequality), you might propose that another system provides a solution (state introduces resource distribution policies), but this doesn’t work if disempowerment occurs in all systems at once.
By activity: Labor (product development, researching, making art), decision-making (investments, life choices, policy, planning), and social participation (relationships, friendships, mentorship, therapy, community).
The disempowerment processes often act on one activity in particular, like standardization reducing decision-making freedom while automation typically impacts labor (though future AI might automate all three). Value and identity are shaped by activities in every category: labor (art, entertainment media), decision-making (counselling), and social participation (friendships, norms).
By type of unfreedom: displacement (being pushed out of roles in work, relationships, or elsewhere), control (having one's beliefs and actions manipulated by external forces), and constraint (limits on what one can do, learn, or say).
There's a qualitative difference between losing power because you're no longer needed (displacement), which doesn't technically restrict your actions, and being directly manipulated or restricted. These apply to individuals but also to whole societies. Constraint can be split further into hard constraints on available options (things being illegal) and soft constraints that merely make things difficult (coordination challenges).
By type of political power: decision-making power (overt, direct power in the hands of whoever decides), agenda-setting power (covert power, wielded by setting norms and values or designing the decision procedure), and ideological power (hidden power, wielded by influencing people's very thoughts and desires).
This is Steven Lukes’s three faces of power, from Power; A Radical View. (A shorter description of the three faces is available here.) We might hand direct decision-making power to an AI, but AIs also reshape our norms and values as we interact with them and each other through media and chat interfaces. AIs becoming superhuman at persuasion is a frightening prospect, perhaps an even more insidious form of disempowerment.
By scale: individual, community, society/humanity.
Individuals can lose power, groups can lose power, or humanity as a whole can lose its ability to steer the future. This division is useful for thinking about trade-offs: a powerful centralized authority might more easily enforce rules that keep humans in control overall, but at the cost of disempowering individuals and groups. The ideal system would empower humans at every scale, both individual freedom and the ability to coordinate in groups, while keeping market pressures and other forces from crowding out human values.
By threat-response capacity: reasoning (“noticing the problem and figuring out what to do”), implementation (“actually doing something about the problem”), and coordination (“effectively collaborating against the threat”).
Proposed by Raymond Douglas and Owen Cotton-Barratt in a LessWrong article, this is a useful framework for analyzing self-reinforcing disempowerment, which grows harder to reverse as response capacity erodes.

Visualization of response capacities (Source: Disempowerment spirals as a likely mechanism for existential catastrophe)

By abstraction level of human influence: direct control (human decision-making, planning, design, idea generation), influence of human values and interests (collective decision-making, cooperation, morality, reciprocity), and the power of human forces in general (culture, memes, creativity).
Individuals can be disempowered, but at a higher level of abstraction, systems can become more or less influenced by human values as such. Strong market forces managed to decouple animal-welfare values from meat production, as factory farming shows. At a higher level still, we can imagine the disempowerment of human forces in general: culture, for instance, which spreads and adapts but always through humans and human interaction, and which arguably becomes less "human" once shaped in the digital world by recommendation algorithms optimized for retention and revenue or influenced by AI content and bot accounts.
By human-system interaction: dependency (how much the system relies on humans to function), legibility (whether humans can understand it and interact with it effectively), and contestability (whether humans can oppose its functions or replace it with something better).
System participants can be empowered by keeping the system dependent on humans, legible, and contestable, while disempowerment is the result of failing on one or more of these.

Concluding Thoughts

We need much more discussion and understanding of what is happening, both with AI and with the rest of society. The first article describing gradual disempowerment was published only in early 2025, and the topic remains in many ways nascent and underexplored. I encourage readers to join the conversation. We need a better grasp of the challenge, and better visions for how to coordinate and steer our society somewhere good.

3