This post builds on much of what Sahil has already written about in his Live Theory sequence, though it approaches things from a slightly different direction. It comes from a broader body of work exploring systems of governance and law that I am working on as part of the High Actuation Spaces project. Thank you to all who have supported my work over the past couple of years, with special shoutouts to Sahil, Matt and TJ. And many thanks to Blake, whose generosity helped me devote myself more fully to this work.

PS: I’m always happy to chat governance, so please reach out if you have an interest in what I’ve discussed here. I’d be particularly excited to talk with anyone who has interest/expertise in the common law and its intersection with philosophy of mind.

Note: I use ‘theory’ throughout as a broad catchall that extends beyond the usual definition of the word.

The problem with ‘dead’ theories

Distributability vs co-creatability

Conventional conceptions of ‘theories’ (very broadly defined as the conceptual tools we use to make sense of and explain the world) are devoid of their own life – they rely on human users as the only source of intelligence to put them into action. These ‘dead’ theories seek to uncover and convey universal principles that apply across contexts. Insights are imbued into static, generalisable artefacts and distributed for mass consumption. This approach allows for theories to be consistently applied and widely distributed. However, it also means their generalisability is contingent upon the erasure of context-specific detail. I like to think of this as a trade-off between co-creatability and distributability.

Theories that embed local insight, for example, rules of thumb or social customs, are highly co-creatable as they depend significantly on the collective input of local actors for their construction. However, they are also inaccessible to outsiders and, thus, not easily distributable. As a result, their scope of coordination is limited to a specific, locally-defined context.

Efforts to coordinate at a larger scale often involve replacing local theories with universally applicable ones – rules of thumb with formal scientific proofs or social customs with legal codes. These universal theories make insights highly distributable. However, they also reduce the scope for the people who use them to contribute to their construction. When individuals belong to a single universally applicable theory structure rather than a patchwork of locally applicable ones, their ability to influence the content of the theories guiding their decision-making is dramatically diminished. Instead, authority to make changes to theories is reposed within a small group of actors such as the academic community or government bureaucrats.

Centralisation and context insensitivity

While this approach to theory-making has certainly enabled coordination at unprecedented scale, it has also meant coordination has necessarily involved a relentless pursuit of centralisation and, correspondingly, an insensitivity to local context. There are two main ways, I think, in which this insensitivity manifests.^[1] These are best understood by considering how a theory responds to situations where its predictions or expectations are contradicted.

• The contradictory observations may simply be removed from the scope of the theory’s application by making de-contextualising assumptions. The assumption used in economic theories that individuals act rationally and have perfect information is a good example. In this sense, insensitivity is a consequence of excluding otherwise relevant context.
• Action may be taken to bring predictions in line with observations. Theories aren’t just descriptive – they can also realise their predictions by motivating changes in the world that remove or avoid contradictory evidence, erasing variation to fit an existing model. In an unblinded experiment, for example, researchers tend to ignore variance and instead interpret observations as validating their predictions. In this sense insensitivity is a consequence of variation in local context being homogenised to fit theory-based expectations.

Sometimes insensitivity to context is an intentional design decision with mostly benign consequences. For example, controlling variables for the purposes of experimentation or prohibiting protected characteristics from influencing employment decisions. However, too often desensitising assumptions are made implicitly or even venerated as emblematic of competence or scientific rigour rather than a necessary but unfortunate limitation of investigative methods. In these cases, a theory’s exclusionary and homogenising tendencies can give rise to great harm. There are many circumstances where this dynamic plays out; one of the most damaging, as James C Scott has written in Seeing Like a State, is when a theory is backed by state power, thus providing unparalleled scope for ‘much of the reality [depicted by the theory] to be remade’ in its own image.

In the 19th-century, for example, the theory of ‘scientific forestry’ motivated German governments to replace old-growth forests with uniform monocultures that were easier to manage and tax.  While this simplifying model initially boosted timber yields, the fragile ecosystem eventually suffered from Waldsterben (forest death). A far grimmer outcome resulted from the Soviet collectivisation of farming. The state's abstract model of industrial agriculture was forcibly imposed on millions of peasants, ignoring their localised knowledge of specific soils, climates, and crops. While this enhanced the state’s control of land and farming, it also led to catastrophic crop failures and a famine that killed between 3 and 20 million.^[2]

Scott’s explanation for this behaviour is, essentially, one of context insensitivity. He argues that states exhibit behaviour patterns where, in seeking to make society governable, they construct synoptic models which lend legibility to otherwise opaque social systems. Faced with the impossibility of comprehensively modelling the world, states resort to coercing the subjects of measurement to conform with their abstracted models. Although this makes societies more amenable to control, it erases context that is only accessible at a local level but which is critical to the system’s continued functioning. Consequently, the structures that enable value creation fail – sometimes resulting in lower timber yields, sometimes resulting in the deaths of millions.

Insensitivity and AI safety

Although the nation-state's monopoly on legitimate violence means it has a unique capacity to wield theories coercively, the phenomenon Scott described is not limited to state schemes.^[3] I would argue they can be grouped into a class of insensitivity (or indifference) risks that also encompass many AI failure modes.

Stable totalitarianism, for example, can be thought of as a failure mode in which AI enables governments to operate in a way that is completely indifferent to the welfare of their citizens. Functional constraints on information gathering and interpretation that previously limited the scope of state coercion are removed – ‘[w]ith modern surveillance to collect information, and modern AI to interpret it, there may be no place to hide’. Abstract models of society can be implemented in their totality without disturbance from unpredicted variations in individual behaviour.

This framing also captures more diffuse failure modes, where this insensitivity isn't driven by a single totalitarian state but by competition between oligopolistic interests. Another (outer) alignment failure story describes a world where  ‘regulations are static…corruption is endemic…’ and where ‘new policies [are being] pushed by ML-enhanced lobbyists’. In this scenario, central coordinating institutions are sensitive to the goals of a small group of powerful actors but are numb to the needs and welfare of the broader citizenry, whose interests are disregarded in the competitive struggle.

Insensitivity risks can also frame failure modes [specific to the operation of AI]. Paperclip maximisers, for example, can be thought of as extreme cases of theory-driven homogenisation: in repurposing your atoms to bring the world in line with its expectations of paperclip abundance, Clippy exhibits a profound insensitivity to your bodily integrity. And AI deception can be thought of as cases of insensitivity via exclusion. Rather than adapting its internal models to align with instructions that contradict predictions, the AI responds by excluding human oversight from the scope of context relevant to its decision making.

Framing these examples as instances of insensitivity failure helps us link them back to a common pathology – approaches to theory making that prioritise distribution of theories embedded within AI models over co-creation of theories with their consumers.

This pathology isn’t unique to AI. A focus on distributability over co-creatability has dominated approaches to collective-meaning making in many areas of human endeavour, and is deeply  embedded into the fabric of our current theory-making tools. Thankfully, however, this may be a relic of our era. Advances in AI provide opportunities to develop new ways of distributing theoretical insights at scale without sacrificing co-creatability found in local systems. This is what live theory seeks to achieve.

Live theory

The relationship between producer and consumer

Note: Credit to Sahil, the original author of this example and these diagrams.

Live theory frames the dissemination of theoretical insights as involving a relationship between a theory’s producer and its consumer. Producers generate novel ideas, and package them into artefacts that can be used to convey their ideas to others. Consumers interact with the artefact to extract those ideas and integrate them into their own world models in terms that are relevant to them. When new technologies are embedded in theory artefacts, the nature of the relationship between producer and consumer can be changed.

As an illustrative example, consider the history of the book as an artefact that allows for authorial insights to be conveyed to readers.^[4] The printing press allowed written text to be widely and consistently distributed. However, readers had no input into any aspect of the book’s construction or development – by the time it came into their possession its text was static and unchanging. All the labour of interpreting and integrating the author’s ideas into terms that are relevant was borne by the reader and, unlike with in-person discursive explanations, the artefact cannot respond or be amended to better suit the reader’s needs and background.^[5] 

The introduction of e-readers and e-books presented new functionality which, for the first time, enabled readers to change the font of the book post-distribution. Rather than being forced to pre-define the formatting of a book’s font, the author/publisher/device manufacturer leaves that decision to the reader. In a small way, readers take part in the construction of the artefact. This may be of particular significance to a reader if commonly used print fonts are harder to read, for example, because of a vision impairment (or even just because of strongly-held font-preferences).

Now consider the possibility for books to be ‘enlivened’ by LLMs. An author of high-school history textbooks, for example, might choose to forgo drafting different textbooks for different grades. Instead, they could distribute a ‘live textbook’ which is usable by students at a range of ability levels. The textbook is co-created by the reader and author when the LLM provides the appropriately targeted text in response to the reader providing information regarding their prior knowledge and understanding.

If necessary, the live textbook may be designed to ensure that readers in the same cohort are presented with a consistent set of lessons. However, even within these parameters the text could be particularised to the individual needs of students by adapting language, providing expanded commentary or using alternate diagrams. Thus, enlivening the textbook doesn’t simply involve the carte-blanche removal of constraints. Rather, it involves the removal of pre-determined constraints. The author has greater freedom (and responsibility) to decide what limitations need to be implemented to preserve the integrity of their writing rather than those matters being fixed by the limitations of the book as a theory artefact.

Theories as actuating potentials

This example helps illustrate some features of theory-making that are particularly relevant to the live theory agenda. First, it shows that theories are enlivened by a co-creative partnership between producer and consumer. If a theory is produced but never used in the real world by consumers there is little to distinguish it from delusion. If I develop a new set of linguistic representations to describe my thoughts but refuse to share this with anyone, it can hardly be described as a language. If I attempted to use these representations to communicate my ideas to others, my vocalisations would be better described as the ramblings of a madman than novel theories.

Second, the example shows how theories act as tools through which potentials (originated by their producers) can be actuated (by their consumers) – my set of symbolic representations becomes a language when it is spoken by others, my arrangements of paint on canvas become art when it is interpreted as such. Importantly, a theory’s potential may originate from and be actuated by the same person at different points in time. For example, journaling involves a process of actuation if it helps connect my past-self to my current-self so as to reinterpret a confusing past experience and develop a better understanding of myself (hence ‘self-actualisation’).

Third, this example shows how technology can increase a theory artefact’s ‘built-in’ capacity to actuate. With a printed text, the theoretical insight it communicates is enlivened when it is instantiated within a reader’s mind. Although a text may be written in a way so as to afford a greater co-creative role for the reader, when there is a need to convey ideas at scale and with precision the theory’s producer must necessarily impose themselves to a greater degree on the co-constructive process. Consider, for example, the difference in interpretive scope afforded to a reader by a poem as opposed to a recipe.

The 'live textbook' example provides an intuition for the possibilities afforded by AI-enabled actuation. Past technological leaps like the printing press primarily resulted in the expansion of theory distributability. AI offers new ways of communicating theories that enhance their co-creatability without diminishing their distributability. This presents opportunities for new approaches to collective meaning-making that  avoid the pitfalls of insensitivity failure and which prioritise the thriving of a theory’s users over that of the theory itself. This is what live theory seeks to achieve.

Live theory and high actuation

Dead theories achieve intersubjectivity through universalisation – they pre-define static formalisms resistant to the influence of local contexts. To the extent that their artefacts are adaptable to a consumer’s needs, those needs must be compressed into pre-defined parameters – the complexities of a student’s learning needs are reduced to their subject and grade.

In contrast, live theories are ‘high-actuation’ – they allow for theory artefacts (such as the live textbook discussed above) to be constructed after the point of distribution. Producers no longer have to coerce their ideas to conform with the limitations of theory-artefacts, and can instead focus on identifying those constraints that are essential to the integrity of their theory. Live theories also make space for consumers to play a far greater co-constructive role as their local context can be integrated into the genesis of the theory itself. Every student can engage with an individually-tailored account of history.

Individualised theory-artefacts may trigger concerns of decoherence – ‘how can there be a shared grounding in truth without a uniform means of interfacing with it?’ However, this concern belies a perspective that underprivileges the co-constructive role of a theory's users. Even with exact uniformity in distribution, every individual who interacts with a theory artefact is a unique mind with their own qualia and internal world models. Even an effort to communicate a simple concept to an audience will instantiate individualised conceptual constructions unique to every audience-member’s internal experience.

The key difference with live theories is that, because they are ‘high actuation’, the differentiation in theory construction is frontloaded into the theory artefact itself. Individuated interpretations are no longer barriers to coherence that need to be minimised as it becomes the job of the artefact to fluidly port insights across different applications. Thus, rather than a weakness that diminishes a theory’s generalisability, sensitivity to local context becomes an enabler of widespread distribution.

In summary, AI-enabled live theoretical artefacts provide an avenue through which locally-adaptive insights can be distributed widely without needing to pre-define the full extent of their application. This provides a framework through which technical risks associated with AI adaptivity can be addressed (beyond the scope of this post).^[6] It also presents a path to new forms of AI-enabled social coordination that do not demand centralisation. It is this possibility that I am most interested in exploring through live governance.

Live governance

Producers and consumers in law

Live governance is a particular instantiation of live theory that focuses on how normative theories of social behaviour, such as described through law, are produced and consumed. It aims to depart from the idea of governance as a tool for control by using AI to enhance the built-in actuative power of governance artefacts. In the same way that AI presents opportunities to narrow the gap between authorial intent and reader experience through ‘live textbooks’, it also presents new opportunities to more closely integrate local experiences of law (and other governance instruments) with how they are created and implemented.

To understand the merits of bridging the divide between the production and consumption of law, consider the divide as it exists today.

• Even in the best-functioning democratic systems, the connection between legislative processes and the aspirations of citizens is tenuous at best. Some politicians may be responsive to the experiences of their constituents, but special interests almost inevitably wield more political influence. Plus, the dynamics of modern legislative processes mean that lawmaking is often driven by the dynamics of political showmanship rather than policy debate.

• The enforcement of law is highly discretionary, and is often driven by socio-political power dynamics. And in the realm of policy-making, bureaucrats often face perverse incentives that result in outputs upon which external reality has minimal bearing.^[7] 

• The costs of legal action are so significant that only a tiny proportion of justiciable disputes are actually adjudicated in court. As such, relative power imbalances are often the primary driver of dispute resolution rather than legal merit.^[8]

I view these shortcomings as a consequence of existing exclusionary and homogenising tendencies of state-backed theories that render them insensitive to local needs and privilege the position of a small group of powerful ‘theory-producers’. These tendencies also go a long way to explaining why our institutions of societal coordination struggle to adapt to the pace of social, political and technological change as it stands.

If we are entering into a period of epoch-defining AI-driven transformation, we need social institutions that are capable of coordinating at pace and at scale.  Whether the threat arises from oligopolies, authoritarian states or rogue AI, our institutions need to be capable of resisting what could be immense homogenising pressure.

However, we must avoid the trap of focusing exclusively on expanding the scale and pace of coordination – this is simply a different way of prioritising distributability and would likely lead back to the familiar, flawed bargain of sacrificing co-creative agency for centralised control. The true challenge is achieving coordination at scale in a way that is sensitive to local context. While this has proven exceedingly difficult in the past, it may soon be possible through the careful application of AI-enabled governance tools.

Possibilities of live governance

As with live textbooks, live governance envisages AI-enabled governance artefacts functioning as an intermediary between the production and consumption of law. If successful, this would mean that from the perspective of the state there would no longer be an imperative to force homogeneity – differentiation in how laws are constructed would not, by default, undermine their integrity. From the perspective of the individual, the apparatus of the state would become less opaque, more malleable and more locally responsive.

As an illustrative example, imagine a world of ubiquitous health-aware AI-assistants. A group of city-dwellers might unknowingly share a common problem linked to the city environment, such as severe hayfever caused by the trees located within a specific park. One city-dweller’s AI companion, noticing the pattern, could connect them with others suffering the same problem. This group could coordinate through their AI companions to model the health impact, analyse the cost-benefit of replanting the park, and generate a policy proposal for the local council. This illustrates a future where policy is not just influenced by existing, well-organised lobbies, but can be driven by context-specific coalitions that emerge and dissolve fluidly.

Extended to AI safety, we might envisage a world where a survey identifying widespread support for AI regulation is not simply an item of evidence for consideration by a small number of interested politicians but instead the spark for AI-actuated collective action. Or where the Intergovernmental Panel on AI Safety reports don’t only reflect a consensus among computer scientists but also integrate economic, sociological and philosophical perspectives into a cohesive whole.  

The possible scope of live governance is far broader than these examples, and encompasses potentially transformative structures (and processes) of law.  While I have confidence that it is a promising framing for future investigation, I also view that its success is dependent on implementation that is nuanced and well-adapted to specific use-cases. For this reason, I will refrain from articulating anything more detailed regarding how live governance might be applied in this post lest I give a false impression of clear solutions. But this is certainly something I intend to explore further.

Aesthetically, I envisage live governance as departing from kafka-esque experiences of bureaucracy and moving toward a coordinative approach that is more akin to a biological understanding of ‘civilisation as an organism’.^[9] In the same way that we don't think of the human mind as exercising totalitarian control despite it ‘directing’^[10] the actions of trillions of independently homeostatic cells, live governance seeks to redefine the nature of governance such that the myth of a centralised 'state-as-controller' can begin to dissolve.^[11] 

My hope is that by leveraging the capacity for AI to actuate otherwise dead theory artefacts, live governance can help develop new modes of social coordination that are both scalable and adaptive to local context. This could lead to systems of government and law that are simultaneously more cohesive and more diverse, enabling societies to adapt to the challenges posed by transformative AI while preserving and enhancing individual agency.

1. ^{^}

   Though these are perhaps more correctly viewed as two versions of the same thing.

2. ^{^}

    See Chapter 1: Nature and Space and Chapter 6: Soviet Collectivization, Capitalist Dreams of Seeing Like a State.

3. ^{^}

   Scott himself acknowledged that ‘the conclusions that can be drawn from the failures of modern projects of social engineering are as applicable to market-driven standardization as they are to bureaucratic homogeneity’.

4. ^{^}

   This example could be extended further back in time to describe the transition from oral traditions (highly co-creatable) to written language (distributable but expensive). However, a truncated example has been provided for sake of brevity.

5. ^{^}

   Those who have struggled with a single, poorly written explanation in a textbook might recognise this pain.

6. ^{^}

    ‘We view these two relationships to substrate (substrate-dependence and substrate-independence) as defining the two dominant research proxies historically used in AI safety research, corresponding loosely to the prosaic and agent foundational camps.

   In order to remain applicable and reliable, our techniques for analyzing and intervening upon the risks posed by AI will likely need to straddle both approaches, neither going all-in on universal invariants nor restricting itself to only localized contexts.

   Instead of creating abstractions that are substrate-independent, we aim to articulate designs that scale specificity directly. This has not been possible before, but we suggest that recent AI advances have made it possible to start incorporating such scaling into our research methodology. This deserves design attention.’ From MoSSAIC: AI Safety After Mechanism.

7. ^{^}

    ‘I spent 10 years of my life writing. I wrote neighbourhood plans, partnership strategies, the Local Area Agreement, stretch targets, the Sustainable Community Strategy, sub regional infrastructure plans, funding bids, monitoring documents, the Council Plan and service plans. These documents describe the performance of local government and its partners. I have a confession to make. Much of it was made up. It was fudged, spun, copied and pasted, cobbled together and attractively formatted. I told lies in themes, lies in groups, lies in pairs, strategic lies, operational lies, cross cutting lies. I wrote hundreds of pages of nonsense. Some of it was my own, but most of it was collated from my colleagues across the organisation and brought together into a single document. As a policy, partnerships and performance officer in local government, this was my speciality and my profession.’ From Public sector porkies – 10 years of lying up the hierarchy.

8. ^{^}

    In common law systems this has flow-on effects because the areas of law that are most litigated end up disproportionately contributing to the corpus of precedent decided by appellate courts.

9. ^{^}

   This reflects an understanding of intelligence as a process that necessarily involves collective decision-making, and which can be used as a framing for ‘systems far beyond familiar animals with central nervous systems’. See Collective intelligence: A unifying concept for integrating biology across scales and substrates.

10. ^{^}

    Even conceptualising this as ‘directing’ makes little sense, since the mind is also billions of tiny animals itself – who is controlling whom?

11. ^{^}

    Note, I don’t view this as involving the dissolution of the state itself.