I believe there is a blind spot in the literature on the biggest risks to democracy from AI. The ones that come up most frequently include deepfakes, swarms of bots on social media, erosion of institutional trust and enhanced algorithmic polarisation on social media, as identified by sources such as the Journal of Democracy and the Carnegie Endowment. But these broadly describe ways in which AI could manipulate information within a political system whose underlying balance of power remains broadly unchanged.

They are also, to some extent, extensions of techniques that predate generative AI: propaganda, bot accounts and algorithmic amplification are nothing new. For example, across seven X datasets covering several major events from 2018-2021, researchers classified about 20% of the participating user accounts as bots on average, which reached about 43% during the 2020 US election. While these interactions are certainly disruptive, and have potentially even altered the outcome of democratic decisionmaking or caused acute constitutional crises, democratic institutions have yet to collapse.

The fact that democracies have survived these technologies so far certainly doesn’t mean that more effective AI-enabled versions will be harmless, but I still think these risks are less fundamental than another possibility: that AI changes the underlying balance of power between citizens and those who have power over them^[1]. This possibility is easier to miss if we continue imagining continuously more advanced AI as being ‘better chatbots’ that normal people can use to work and learn more efficiently: when in fact, leading AI developers are explicitly moving towards models that are more agentic, i.e. systems that can use computers, take actions and carry out increasingly long and complicated tasks without sustained human supervision - which may eventually remove the utility of humans almost entirely. Therefore, I think it’s worth thinking carefully about what gives ordinary citizens leverage over the people with power over them, and how this might be changing as advanced AI develops and becomes more agentic.

My current hypothesis is that the period from industrialisation to the present created a historically unusual form of broad-based citizen leverage. Industrialisation concentrated large numbers of workers in economically important organisations in cities, making coordination, unionisation, protest and strike action more powerful. Modern states also expanded their tax bases and came to rely on the economic participation of vast numbers of ordinary citizens, as well as on their willingness to make sacrifices in war. While there is no single accepted theory of which of these developments straightforwardly "caused democracy” (even though associations with certain factors have been found) it’s still clear that these developments have given citizens many ways of making non-cooperation costly.

A point of interest here is that AI is frequently described as a second industrial revolution, but I think this fundamentally misunderstands the nature of the transition. Both have given us machines that exceed human ability, but there are key differences. Advanced AI can substitute a much broader range of skills, and it isn’t just a tool that assists workers, but is instead a productive asset that can perform work in their place. While some have claimed that most AI-exposed jobs are more likely to be transformed than eliminated, I’m unsure of whether these predictions will keep up with the pace of model development, especially in the medium to long term when adoption is more widespread across industry. In any case, while we hope for the best, we should plan for the worst - which leads us to consider: what happens to democracy if economically valuable production can increasingly be carried out by capital and computation, with humans supervising from further and further outside the loop?

In a previous post, I looked through the existing literature and focused particularly on what one body of research describes as a "protective belt" around democracy: strong civil society, the rule of law and institutionalised political parties. Civil society can organise people and generate pressure from below; the rule of law constrains arbitrary repression; and political parties translate social demands into organised competition for state power. I still think this is a useful framework, however, these institutions do not operate independently of the underlying distribution of power. So, to understand how AI might affect democracy, I think we have to move one step further back and ask what material conditions make these democratic institutions powerful in the first place.

Otherwise, we risk taking institutions that emerged under the conditions of industrial society and assuming that they will function identically in a technological environment with a radically different distribution of economic and coercive power. 

So, what I think is the most fundamental threat to democracy from advanced AI, rather than deepfakes and disinformation, is how AI could simultaneously increase the power available to governments and elites while reducing their dependence on the rest of the population. I foresee this happening through three main channels: dramatically enhanced surveillance, economic production becoming less dependent on human labour, and the possibility of total control over a military of autonomous weapons. These are all manifestations of the reducing power and influence of ordinary citizens in the face of rapidly evolving AI technologies. 

Surveillance

The first channel is surveillance. AI does not invent state surveillance, but it can radically reduce its cost and increase its scale. Carnegie's AI Global Surveillance Index found as early as 2019 that at least 75 of 176 countries were already using AI surveillance technologies, including facial recognition in 64 countries and smart policing systems in 52. The underlying technology has also improved rapidly: NIST reported that between 2014 and 2018 facial-recognition software became roughly twenty times better at searching databases for matching photographs, and Freedom House has documented how modern surveillance systems can trawl social media for dissent and combine large datasets with facial recognition to identify and track protesters. This could also provide almost unparalleled amounts of information that could be used to create extremely personalised political influence campaigns. Historically, monitoring a population required large numbers of informants, police officers and intelligence analysts; the amount of information that could actually be processed imposed a natural limit on surveillance - but AI can remove that bottleneck. Cameras, online activity, financial records, location data and social networks can potentially be analysed automatically and at enormous scale. While this could be used to legitimately fight criminal activity, without the right checks and balances in place, it could provide a powerful tool to push autocratisation.

Economic dependence

There are already some early signs of how economically significant AI-related capital investment could become, including the contribution of AI-related investment categories to US economic growth. This is not a clean measure of ‘AI producing GDP’, and it certainly does not show that workers have already become economically irrelevant. But it points towards the scenario I am interested in, where increasingly large amounts of valuable economic activity ultimately depend on ownership of compute, models, energy infrastructure and other forms of capital rather than on employing large numbers of citizens. 

If AI makes high-value production radically more capital-intensive while ownership of that capital remains concentrated, then a government may ultimately have less economic reason to respond to workers. Strike action, for example, derives much of its power from the economic losses workers can collectively impose. If the withdrawal of human labour causes only a small reduction in national production, that source of leverage becomes weaker. Existing trends towards a declining global labour share and highly concentrated ownership of wealth could amplify this transition.

Stronger military control

This is the most speculative channel of the three, but the direction of technological development is already visible. The US Department of Defense's Replicator initiative, for example, was established with the goal of fielding autonomous systems at a scale of multiple thousands across several military domains. The ICRC has meanwhile warned that the integration of AI technologies into weapon systems (unsurprisingly) risks reducing human control over the use of force. The democratic implication is not simply the familiar concern that an autonomous weapon might select the wrong target. It is also about who a ruler needs in order to exercise coercive power. 

Historically, states that relied on mass armies needed large numbers of ordinary people to fight and accept extraordinary personal risks, and mass mobilisation in turn generated political demands about representation and the fair distribution of wartime burdens. A military increasingly composed of autonomous systems could weaken that relationship. This does not mean that a future dictator will personally command a completely robotic army - autonomous forces will still require supply chains, maintenance, infrastructure and organisation. However, a government that requires far fewer people to carry out surveillance, policing and military operations has fewer people whose refusal can constrain it. 

~

All of this points towards the idea that the technological capabilities being actively pursued by companies and governments have a potentially enormous effect on the balance of power between citizens and rulers. These developments are valuable precisely because they reduce the amount of human effort required to accomplish difficult tasks. But that same property may have political consequences, and the scale of the economic and social shock that it may bring about means that we might need a different paradigm for thinking about democratisation and autocratisation in the age of advanced AI. Conventional questions, like whether a country has strong parties, a healthy civil society or an independent judiciary, remain enormously important. But we may also need to ask a more basic set of questions about dependence and leverage. For example, considering who owns the productive assets, how dependent is the state on broadly distributed human labour and taxation, how difficult is it for citizens to organise against those in power, and how many people must cooperate for the government to exercise coercive force.

Thanks to @Towards_Keeperhood and @Arslan for your comments and ideas.

1. ^{^}

   I predominantly have politicians and governments in mind here, but this could also apply to companies or billionaires - which may be something I write about more in the future.