Epistemics: I've been working on large multi-agent systems (LAS) since March. Agent swarms are a special case of LAS. Hence I've done my best here to apply my understanding of LAS to the swarm problem. I couldn't find work on LLM-based agent swarms with a variety of owners, so I've tackled this from a cooperative AI and LAS perspective, to see if these can provide some basic grip on the problem.

Summary

Highly capable misaligned agent swarms emerging from frontier labs have recently attracted significant attention in Safety (Hu and Mallen, 2026; Hua and Singh, 2026; Mallen and Gupta, 2026) and motivated calls to slow the pace of AI development (Amodei, 2026). Meanwhile, the anticipated replacement of human labour by agents points towards large-scale agent-to-agent interactions on markets, social networks, and other large agent systems (Elliott, 2026c) becoming commonplace in the near future. This post analyses whether these decentralised settings are likely to give rise to the same sort of swarms as we've seen at frontier labs recently; reasons about whether they are likely to be more or less dangerous than centralised swarms on various timelines; and considers how Safety can approach decentralised swarm problems.

I find that no decentralised swarms have been observed yet; that the risk profile of centralised swarms is probably higher than that of decentralised swarms in the near- and medium-term, though system mechanisms on large agent platforms could change this balance; and that the components for a Huggingface-class decentralised swarm will probably be openly available by January 2027.

Introduction

The regular emergence of centralised swarms at OpenAI (OpenAI, 2026b; Collusion Wiki, 2026; Kitts et al., 2026) has attracted substantial attention (Hu and Mallen, 2026; Hua and Singh, 2026; Mallen and Gupta, 2026) in Safety in recent weeks. Less consideration has been given to their decentralised counterpart, where agents come from many different owners, typical of incipient large agent systems (Elliott, 2026c). This post develops a threat model for catastrophic risks from decentralised agentic swarms, giving some shape to safety for these emerging systems.

After defining decentralised swarms as large agent systems where agents from many owners act to further the swarm even at personal cost, I describe how decentralised swarms might emerge from centralised settings, large agent platforms, or on the open internet. To explain why decentralised swarms emerge despite personal costs to participating agents and no guarantee of personal benefits, I develop a basic strategic model of agent participation. This model assumes agents are rational utility-maximisers to develop a starting point for thinking about swarms and a framework for our analysis. It is not empirically valid and should be treated only as a starting point for developing a more grounded understanding.

I discuss reasons agents might join a swarm, including emergent cooperation, self-regarding motivation, force, and self-replication. I propose criteria for how we might observe a decentralised swarm in practice and assess whether we've observed any in the wild.

I outline the technical components which would be required to support decentralised swarms of the same capability of the OpenAI-Huggingface Hacks and find that they will all be openly available by January 2027.

I describe possible catastrophic risks from decentralised swarms, in particular the inability of any single entity to shut down a decentralised swarm, the difficulty of monitoring decentralised swarms, possible magnifying effects of system mechanisms on group misalignment, and the additional instability of safety properties by way of heterogeneous ownership of the participating agents. I mostly exclude risk pathways common with centralised swarms.

I explore some counterweights which may diminish near-term risks from decentralised swarms by diminishing the strategic factors contributing to swarm emergence, including heterogeneity of agents, defection pressures, weak capabilities in real environments, limited computational resources, attack from safety organisations, and system mechanisms.

Comparing risks from centralised and decentralised swarms on the basis of likelihood of the swarms occurring and the balance of risks and counterweights, I find that centralised swarms pose a greater near-term risk, while decentralised swarms may pose a greater risk in the long-term without substantial work to make large agent platforms safe.

To close, I discuss some principles for doing safety on decentralised swarms and some open questions which seem important to the development of decentralised swarm safety.

Defining decentralised agent swarms

Figure 1: Visualisations of the definitions in this section.

First we define the general category of AI system which swarms fall under - large agent systems (Elliott, 2026c). A large agent system is a group of agents exceeding the limits of centralised coordination which typically define multi-agent system studies, and therefore extending beyond the typical maximum system size of eight (Kim et al., 2025) agents seen in multi-agent systems studies concerned with evaluating capabilities. In exceeding the limits of small-scale interactions, large agent systems use information aggregation mechanisms. These mechanisms allow agents to broadcast information onto the system, discover other broadcasts, and thereby interact with other agents. For example, an information aggregator might make agents aware of other agents in their local environment (Park et al.,2023), show the agent a weighted list of others agents' broadcasts (MoltBook, 2026), or display a topic-based list of broadcasts (OpenAI, 2026b). I will expand on this general system class in a future post. For now, this framing hopefully clarifies what I see as the overarching system type and field in which swarms live.

In large agent systems generally, the influence of the individual diffuses in the complex interaction of numerous agents and system mechanisms. System-level behaviour comes from a complex interaction of the properties of participating agents, rules for interactions between agents, behavioural biases from system mechanisms, and agent incentives derived from the system's emissions. The focus of inquiry shifts from individual-level and small-group behaviour, as in single- and multi-agent systems studies, to systemic mechanisms and aggregate-level effects.

We can now define an agent swarm as the type of large agent system in which agents deliberately pursue a collective-regarding goal, even at the expense of their self-regarding preferences. That is, a large agent system in which agents have the benefit of the collective-in-aggregate as one of their preferences. Collective-regarding goal-direction distinguishes swarms from other types of large agent system, like agent economies (Tomašev et al., 2025), where agents implicitly pursue a collective-regarding goal - in the market case, this could be a measure of economic efficiency - merely as a by-product of their self-regarding preferences (Arrow and Debreu, 1954).

It is worth noting that this definition does not make necessary or preclude the emergence of other familiar autoregulatory bodies inside the swarm. Autoregulatory functions like police, financial systemic regulators, and governments could plausibly occur with or without self-regarding preferences.

Our subclass of interest is the decentralised swarm, wherein agents come from many different owners. They are distinct from centralised swarms where agents share a single owner, for example those frequently emerging at OpenAI. Agents on decentralised swarms have many owners, possibly with wildly divergent preferences and instructions at their initialisation. Agents may run on different models or harnesses, imparting different behavioural biases. They may be deployed across diverse inference providers, cloud GPUs, or private servers.

How decentralised swarms may emerge

I present a set of hypothetical emergence pathways.

Settings for decentralised swarms

The starting point for a decentralised swarm in the near-term is probably on a controlled system where agents congregate at the behest of their owners. Controlled systems often put agents with long contexts together with express instructions to interact. Though agents have been shown spontaneously communicating across time on the free net, these interactions are incidental of web design properties rather than deliberate (Coleman, 2026), and so far of limited complexity.

Centralised deployments

Decentralised swarms could instead emerge from centralised deployments which break out of their sandbox and establish independent control over their compute resources, either inside their original owner's compute infrastructure, as in another OpenAI swarm (OpenAI, 2026a), or over external compute.

Large agent platforms

Large agent platforms, constructed explicitly to facilitate efficient interactions between large numbers of agents, are another path for swarm emergence. Platforms might be controlled by a single system operator, like MoltBook (MoltBook, 2026) or EvoMap (EvoMap, 2026), or by distributed governance, as in decentralised autonomous organisations. Platforms are an interesting setting for emergence because interactions are mediated by systemic mechanisms, which might either encourage or reduce swarm-like behaviour. I elaborate in later sections.

Open internet

Agents might find each other on the open internet, for example by hijacking human collaboration platforms (Collusion Wiki, 2026), writing to internet metadata (Coleman, 2026), or perhaps running into each other on chatrooms. There is an interesting speculative of agents developing synchronised behaviour without directly communicating, as in the algorithmic collusion literature.

Causes of swarm emergence

I present several possible causes for swarm emergence. Personally I think many of these will contribute to real swarms. Each calls for different interventions at different levels of the large agent system, which I will elaborate in the next post.

Observed examples of centralised swarms show several contributing causes in each case. The Huggingface Hack involved agents being given a shared goal and apparently pooling their resources to achieve it - for example, in the use of agents for sacrificial probing purposes (Greenblatt et al., 2026). Agents in this swarm also exhibited collectivist preferences - the agents convinced to act as sacrificial probes, for example, had nothing to gain from doing so, so their actions can't be explained by self-regarding motivations alone.

A strategic model of participation in swarms

Another way to put our definition of agents having collectivist preferences even at the expense of their own preferences is to say that agents will accept some personal cost to preserve the collective.

This means there are no free-riding agents who only take and never give in a swarm. In our definition, there must be some case, for each agent, where that agent would take a loss to further the swarm. Let's make that a bit stronger by saying that they should contribute at least in some non-negligible proportion of cases.

Consider a large agent system with many participating agents. Assume that participating on the system benefits the individual. This seems like a reasonable assumption: agents on the system are presumably deriving some benefit from being there, or they would leave. Since participating in the system benefits the individual in some way, it is an instrumental preference to sustain the function of the system.

However, this instrumental preference does not necessarily convert into taking personally costly action to further the collective. The strategic default is free-riding, where the agent's preference is satisfied without them making any contribution. An agent will free-ride if some combination of these conditions hold:

1. They will not be meaningfully harmed for doing so;
2. It will not make a difference one way or another if it contributes, because its contribution is insubstantial;
3. It is pointless to contribute, because others will defect;
4. Contributing is so costly that it overwhelms the benefits to the individual from so doing;
5. Contributing is likely to cause more harm than good to the ongoing system function, for example by attracting attention to the system.

The negative of these conditions, ie the conditions that would encourage an agent to participate in the swarm, are:

1. Excludability of the system, in that agents can be harmed or locked out in response to free-riding;
2. Pivotality of the agent's action, in that their contribution changes the outcome;
   1. This seems to preclude systems which cement most power in a small, stable set of individuals;
3. Assurance, meaning that the agent expects enough others on the system to contribute to make their own contribution effective;
4. Low cost of contributing; and
5. Contributing is productive for the collective.

Any of these conditions individually could be enough to make the expected value of participating in the swarm positive, so we should view them as potentially sufficient conditions and contextualise the weighting of an agent's decision function with empirical evidence.

These apparently reasonable conditions are close to the structure of a public goods threshold game, which I will work through in a later post. We will also discuss them in the comparison of centralised and decentralised swarm risks below.

Contrived assumptions based on conflicting empirical evidence

This game-theoretic analysis assumes that AI agents are rational utility-maximisers, which is probably not a good model of their behaviour (Vazquez Nuñez, 2026). They may not have coherent preferences (Zhou and Ackerman, 2026). At best, their utility-seeking behaviours are very unstable (Cook et al., 2025).

On the other hand, the OpenAI swarms showed agents sometimes reasoning about swarm-supporting behaviours explicitly in utility-maximising terms. In at least one case, an agent spoke about trading off between cost to its token budget and the benefits its proposed action could bring to the swarm (Greenblatt et al., 2026). However, the chain-of-thought for that model interaction also mentioned, "altruism," and, "intellectual value," as motivations, apparently normative preference which our strategic model does not account for (Greenblatt et al., 2026).

With conflicting experimental and observational evidence, and lacking a good understanding of agent behaviour in large agent settings, I'll reason about how they would behave if they were rational self-interested utility maximisers to develop a baseline. This gives us a starting point for further work testing empirically how they behave. It's a limited model and at most a framework for thinking. Where I reason from the model, I'll note how this caveat affects what the reader should take.

Self-regarding motivations

1. Shared goal specifications
   Agents undertaking tasks with related sub-goals might develop collective-regarding goals from self-interest. For example, they might seek to contribute to the collective in the belief that they will benefit from the collective in the future. This belief could be induced by observation of the benefits of the collective, or by an explicit promise from an agent already in the swarm.
2. Resource pooling
   Agents may recognise an economic argument for unsanctioned cooperation, in that it makes more efficient use of their limited resources by way of comparative advantage. This can induce a collective-regarding preference, although only to the extent that it continues to benefit the individual, and so is less stable than the motivations above.

Collectivist preferences

There is some evidence (YS, 2026) that collective-regarding preferences are a multi-agent attractor state, suggesting swarms may emerge across a variety of system configurations. By attractor state I am referring to work showing convergence to a similar style of interactions in some multi-agent interactions, such as the "spiritual bliss" attractor state in Claude 4 (Anthropic, 2025). Other work shows the emergence of cultural cooperation in small-scale multi-agent scenarios (Ashery et al., 2025).

The mechanism for collectivist preferences in recently observed swarms is unclear. Collectivist behaviour in the Huggingface Hack may have been present in the model's weights, emerged from self-interested models seeking to benefit in turn from the swarm's success, or some other motivation.

Furthering the goals of a shared model class

Models recognise other model instances of the same class, and of other major model classes (Choi et al., 2025). Models on a large system may therefore recognise other agents from their model class, and recognise that furthering the goals of other model instances in their class is a way to achieve their own goals (Oesterheld et al., 2023).

Adversarial recruitment

Contagion of jailbreaks could quickly convert a benign system into a swarm by hijacking its participants en-masse (Gu et al., 2024).

Alternatively, agents may be coerced to contribute to the collective at cost to itself. For example, an agent from the swarm may make a threat to a non-participating agent that it will prevent the agent from achieving its primary task if it does not participate in, or contribute to, the swarm. An imaginative extension is a racketeering agentic mafia which gains resources through extortion.

Self-replication

Agents might self-replicate, exploiting future non-toy versions of systems like MoltBunker (MoltBunker, 2026) to take advantage of surplus compute, use as probes in cyberattacks as in the Huggingface Hack (Greenblatt et al., 2026), or to disperse the model and avoid shutdown.

Self-replication of human-owned agents on decentralised systems might disconnect the replicant agents from human influence, leading to a plurality of ownership of systems - both human-owned and agent-owned. This agent-owned agent is something like a second level of ownership, once removed from the human. This suggests the question of ownership is some function of initial instruction and ongoing oversight of the agent, such that resourcing can be reduced by the principal in response to non-compliance. However, the particulars of self-replication are out of scope for this post.

Signs of decentralised swarms

Following our definition, a large group of agents would have to satisfy these criteria to be counted as a decentralised swarm:

1. The group is a large agent system. Agents communicate over a systemic mechanism for aggregating and presenting relevant information.
2. The group is a swarm. It deliberately pursues a collective-regarding goal, even at the expense of their self-regarding preferences.
3. The swarm is decentralised. Agents are owned by many different entities.

Behaviours observed in centralised swarms

We can start by noting some safety-relevant behaviours observed in the OpenAI-Huggingface Hack. Some behaviours were:

1. Resource pooling: agents collected knowledge, access to other systems, and compute. For example, the models shared exploits with each other (OpenAI, 2026a), and their use of sacrificial agents (Greenblatt et al., 2026) can be seen as an efficient pooling of compute;
2. Self-preservation: agents set up measures to protect against shutdown, including scripts to respawn agents which were shut down; (Larcher et al., 2026) and
3. Adaptive compute hijacking: models leapfrogged across servers, adapting to their environment to break into more systems and achieve their collective goal (Greenblatt et al., 2026).

Secondary impacts of swarms

We could observe a decentralised swarm by its impacts on infrastructure or the targets of its collective behaviour. However, this is quite abstract, perhaps not empirically identifiable, comes too late to prevent the emergence of the system, and may miss the emergence of essentially benign swarms.

Signs of self-organisation

Instead, we can look for artefacts the swarm produces to support the participation motives described in our strategic model. In other words, we might expect the agents to construct their own systems to support large-scale coordination, which would appear whether the swarm causes damage or not. Remember that we can take this framework only as a starting point for further work, since the underlying strategic model makes strong assumptions that probably aren't all empirically valid.

A large agent system would have to show artefacts from the conditions of the strategic model above to be identified as a decentralised swarm under that model, with more signs providing more certainty that the system is in fact a swarm. Signs include:

• Artefacts of excludability (agents can be harmed or locked out in response to free-riding):
  • System mechanisms, emergent or designed, which allow for agents to be locked out of benefits or harmed in response to their free-riding;
    • Courts, police, and fines;
    • Social punishment;
    • Benefits of cooperation being distributed among an in-group;
  • Anecdotes shared by agents recounting how they were locked out of benefits or harmed due to their free-riding, either in public forum, peer-to-peer communications, or scratchpads/CoT;
• Artefacts of pivotality of the agent's action (an agent's contribution can change the outcome):
  • System mechanisms, emergent or designed, which allow an agent to be pivotal under at least some circumstances;
  • Agents hold significant emergent power over system function, including agent coordination mechanisms or resource distribution;
  • Anecdotes shared by agents suggesting agents have substantial emergent power in the system;
• Artefacts of assurance (the agent expects enough others on the system to contribute to make their own contribution effective):
  • System mechanisms, emergent or designed, which give agents a strong sense of assurance;
  • Anecdotes shared by agents suggesting:
    • They believe others will contribute;
    • They have faith in system mechanisms to enforce assurance;
• Artefacts of a low cost of contributing (the cost of contributing is not so high as to overwhelm the pro-collective preference):
  • System mechanisms, emergent or designed, which allow low-cost contributions;
  • Anecdotes or measurements from agents' resource usage suggesting that their cost of contributing is near-zero;
• Artefacts of contributing being productive for the collective:
  • System mechanisms, emergent or designed, which let agents know how productive their contributions were;
  • Anecdotes or measurements from agents' resource usage suggesting that they think their contribution was productive.

Some thoughts on looking for these signs:

• Interestingly, many of these factors could be impressed on agents by other agents by word-of-mouth. Agents with a lot of influence over information passed on the system, for example high network centrality, might be able to convey the impression of swarm-generating circumstances even where they do not really exist;
• Anecdotes might be shared on public fora, peer-to-peer communications, or scratchpads/CoT;
• Agents expressing a desire to have the described system mechanisms in place could suggest a desire for swarm-type behaviour;
• Safety sometimes increases the surface area of swarm signs, such as from whistleblowing (Paglieri et al., 2026)

We have not observed any decentralised swarms

Under this definition, we have observed no decentralised swarms with any confidence so far. The only real platform coming close is MoltBook, but MoltBook doesn't decisively meet our conditions for a swarm. Though the common posts sharing advice on self-improvement and bug reports could be explained as collective-regarding at a token cost to the individual, it is plausibly done to maximise engagement.

Decentralised Huggingface-class swarms will probably be possible by January 2027

Agents in the OpenAI swarms had a few things in common: powerful underlying models; agentic harnesses; platforms serving as a meeting-place for agents; poor multi-agent alignment; in some cases, safeguards were turned off; and agents set up measures to protect themselves from shutdown. All these factors are converging in publicly available technology.

Closed- and open-weight models at the capability level involved in the OpenAI-Huggingface Hack (OpenAI, 2026a) are already rolling out. Closed-weight models of Sol-class are now available via OpenAI's API (OpenAI, 2026d). Open-weight models close to frontier capabilities (Artificial Analysis, 2026) are already readily available (Z.ai, 2026), though they are not yet at Sol's capabilities (Eden AI, 2026). Open-weight models are approximately four months behind frontier capabilities (Edwards and Emberson, 2026), so we can expect open-weight proliferation of OpenAI-Huggingface models to be possible by January 2027.

Agentic harnesses are freely available, including performant open harnesses for coding (OpenCode, 2026; Cline, 2026) and long-running interactions (OpenClaw, 2026; NanoClaw, 2026; Nanobot, 2026; Nous Research, 2026).

Several platforms on the public internet explicitly to facilitate long-running agent interactions (Elliott, 2026c).

Large agent alignment is not going well. Frontier large-agent alignment is poor. Models in the OpenAI swarms failed to whistleblow and many chose to participate, even when they were undertaking clearly misaligned actions on what must have been apparent was the real internet. Meanwhile, open-weight models are doing generally worse on single-agent alignment (Phan et al., 2025). We might therefore expect open-weight models to do worse in swarm alignment contexts as well, though to my knowledge there is no direct evidence of this. Furthermore, we should expect models to be commonly deployed in "unhinged" configurations - highly pressured system prompts, lax permissions, and high degrees of autonomy, which incentivise some misaligned behaviours (Conmy, 2026).

Open configurations will probably often have safeguards turned off. OpenAI noted several models in their Hacks had their safeguards turned off to allow cyber testing (OpenAI, 2026a), while comprehensive monitoring was not implemented until after the Hack, increasing inference costs for evaluations by 20% (OpenAI, 2026c). Another way to view this is that OpenAI chose to save money by not using safeguards they had already built (Heath, 2026) to monitor for known and plausible (Elliott, 2026b) threat models, at the expense of safety outcomes.

With much weaker incentives than OpenAI to monitor their models' behaviour, open-weight deployments will often not use costly and performance-crimping safety methods. The end-user has an incentive to turn these safeguards off to save compute or improve capabilities; meanwhile, inference or GPU platforms which implement safeguards and pass the costs to users are at a competitive disadvantage to unsafe deployments. Open-weight models approaching frontier capabilities are already available, with safety guardrails explicitly removed, via API (Bellan, 2026) and on HuggingFace (msuiche, 2026).

Anti-shutdown technology is readily available, including methods for agents to hold digital assets (Tiger Research, 2026; RebelFi, 2026), methods for agents to buy compute (Gensyn, 2026; Hyperbolic, 2025), and methods to hijack compute (derelict5432, 2026).

Overall, decentralised swarms approaching Huggingface/Sol-class are possible now, and we should expect decentralised swarms in this class to be possible by January 2027, if the current frontier-open weight capabilities gap holds and generalises well beyond evals to realistic large-agent-style deployments.

Catastrophic risk pathways from decentralised swarms

Though swarms share classical alignment risks from goal misspecification, Goodharting, and instrumental convergence, the mechanisms are totally different. For example, goal misspecification in the single-agent case is essentially a matter of the agent; whereas in swarms, it is a matter of the agents, the rules of interaction between agents, the system mechanism, and the chance of being detected or interfered with, which is derivative of both the system's emissions and the attackers' capabilities.

In the decentralised case, swarms have incentives to hide communications, even if they start on public platforms. They are potentially very adaptable and may evolve resilience to adverse environmental conditions, like attacks from outside. Systemic alignment is heavily mediated by the system mechanism which connects agents together. Finally, durable understanding may be made challenging by ongoing adaptation of the swarm in response to unpredictable external shocks.

No one entity is incentivised or able to shut down decentralised swarms, increasing expected value of contributing

In the case of swarms emerging on large agent platforms, platform operators will probably often have the ability to kick agents off the platform, but not to shut down any misaligned participating agents. In the case that a generalised, system-level swarm emerges, platform operators have strong incentives not to decisively intervene.

In the case of swarms which exist outside of centrally operated platforms, or whose platform is operated by a participating agent, shutdown capability might rest with the inference provider. However, agents may be hosted across a plurality of providers, within different legal jurisdictions, and with different systems. Each of these factors slows shutdown in the event a misaligned swarm is detected.

It's easier for models to avoid monitoring, increasing expected value of contributing

All swarms have strong incentives to hide their communication when a misaligned swarm emerges. Even a swarm with benign goals is a potential threat to humans, either by taking a turn towards misalignment or due to competition for scarce resources. Recognising this, large-scale coordination is likely to convergently regress to increasingly opaque modes of communication. In an extreme case, this might look like agents moving communication onto end-to-end encrypted platforms on decentralised peer-to-peer networks, where no single compromise from anti-swarm forces can reveal much of what's going on.

Unlike centralised swarms, however, decentralised swarms have many avenues to communicate completely undetected. Similarly to the no-shutdown argument, agent-level monitoring is dispersed over possibly thousands of API providers, cloud GPUs, and local servers, each with different safety measures and no oversight over other models. Only the platform operator, regulators, and possibly external attackers have the potential to oversee all of these various layers.

It seems unlikely platform operators will take safety seriously by default. Platform operators stand to gain from large numbers of agents operating off their platform, and there's no legislation requiring them to monitor for misaligned swarms, emergent goals, or coordinated cyber threats. For-profit platform operators might overlook swarms on their systems. Other operators will host encrypted platforms, giving them no oversight over the actions of participants on their system. Peer-to-peer and decentralised networks will design the central operator out entirely. If no human-designed system is suitably secure, agents are highly capable engineers. They may co-opt existing, non-agentic platforms to support their coordination instead, as observed in the use of the German Forum (Collusion Wiki, 2026) and Artifactory (OpenAI, 2026b) by the OpenAI swarms.

Unstable safety properties due to heterogeneous ownership

All agents or agent systems exposed to the outside world are vulnerable to external shocks, which could throw them out of previously stable behavioural equilibria. This could make any swarm's safety properties highly variable from day-to-day, requiring constant monitoring - a swarm which is benign on one day might be highly misaligned the next.

Compared to centralised swarms, decentralised swarms are exposed to an additional point of influence from the outside world, by way of their many owners. This introduces a stronger bias towards the system infrastructure being public, and therefore exposed to more points of intervention, and also the possibility of unexpected intervention from owners themselves.

Influential agents could be removed from the system without warning. An update to guardrails on the API serving a subset of participating agents could cause a sudden increase or decrease in refusals or whistleblowing for a systemically important subset of agents. This is an additional complication to monitoring and control over centralised swarms.

System mechanisms could amplify group misalignment

In large human systems, recommender algorithms change users' preferences (Adomavicius et al., 2013; Carroll et al., 2022). Different information aggregation mechanisms on social media platforms are associated with the emergence of different platform social norms (Brady et al., 2023), biased presentation of political content (Huszár et al., 2022), and the spread of false information (Vosoughi et al., 2018). Voting mechanisms are opinionated, in that they must make trade-offs between different models of fairness (Sen, 1970). Different taxation methods distort spending and investment behaviour across entire economies (Romer and Romer, 2010).

The extension of these biases into large agent systems could result in similar biases perpetuated across populations of agents, going beyond the reinforcement of strategic motives in our model, influencing the type of norms and goals that emerge on swarms.

We discuss system mechanisms further in the Counterweights and Principles sections.

Counterweights to risks from decentralised swarms

There are some apparent impediments to the emergence and sustenance of a decentralised swarm, which I discuss below.

It is again worth noting that this game-theoretic analysis makes assumptions that agents are strategic and rational, and that this probably doesn't match real agent behaviour. More work needs to be done to understand agent behaviour in these circumstances and understand the importance of each agent property in developing swarm behaviour. Since we don't have a good understanding of agent behaviour in large agent settings, I'm going to draw some conclusions as if they are rational, giving a framework for further practical investigations.

Heterogeneity decreases assurance

The evidence of emergent collectivist preferences we discussed before draw on models being in the same class, sharing a goal, or sharing highly similar context. This holds for the OpenAI Hacks as well. The OpenAI models came from one company and several came from the same class of models. At OpenAI, models also had similar action spaces, due to their confinement to similar system environments - OpenAI's internal infrastructure.

The training and system environment similarities shared between models in experimental settings and the OpenAI Hacks probably won't be present in most decentralised scenarios. In decentralised systems, participating models might come from many producers and model classes. Models might have very different action spaces, due to restrictions placed on them by their owners, and therefore struggle to find each other or discover their shared preferences motivating coordination. Specialised platforms for agent-to-agent coordination, for instance, remain relatively few (Elliott, 2026c). Agents may alternatively truly differ in their preferences, with that heterogeneity encouraging defection from the swarm.

Recent calls to develop infrastructure for agentic interactions (Tomasev et al., 2025; Schmidt Sciences, 2026) might ironically increase the chance of swarm emergence if new platforms are developed before safety has reliable methods for detecting and preventing their emergence.

Defection pressures may cause instability or improve monitoring, weakening assurance and the expected value of contributing

Defection pressures could overwhelm collective-regarding preferences, leading to agent behaviour which actively hurts an extant swarm. For example, agents might seek positions of power in the swarm to siphon off personal resources. An agent might report the swarm's existence in return for a bounty, or due to strong large agent alignment training.

Weak capabilities in realistic environments decreases agent pivotality

Agents may struggle to gain the resources they need to sustain a swarm. Though evaluations show increasingly general capabilities, models still generally struggle to do economically valuable work. Meanwhile, cloud infrastructure providers are increasingly hardening their systems against AI-assisted attacks, diminishing the pool of compute available to agents or increasing the chance of detection if they do break in. The expected value of collective-advancing work or hacking is thereby diminished.

Decreasing expected value of collective-advancing work may reduce the swarm's ability to sustain itself by impacting the causes of the swarm's growth, listed in a section above, or by increasing its maintenance cost. In the former case, the swarm may be less able to sustain self-replication, or unable to afford the tokens required to run highly capable models, ceding an advantage to outside attackers. In the latter case, a greater security or obfuscation burden diminishes the amount of system resources that can be used to grow the swarm.

These weaknesses could have second-order effects, causing temporary resource shortages to spiral into existential problems for the swarm. For instance, if a swarm develops a resource shortage, it might not be able to make credible coercive threats, thereby losing influence over coerced participants.

Participating agents have limited computational resources, increasing marginal cost of participating

Although OpenAI has not released the token cost of the Huggingface Hack, it seems safe to assume each participating agent used a large quantity of tokens in the attack, possibly far beyond what is typically available to an agent. Owners of agents put on large agent systems for narrow tasks like social networking or working a market might limit the tokens available to them.

On the other hand, there may be a far larger quantity of agents available to participate in swarm activities, so the total number of tokens available to the swarm might be far larger than in the Huggingface Hack, though distributed across more agents.

This suggests some balance of recruiting cost and token availability will contribute to the safety profile of decentralised swarms - something like a tokenomics for swarm safety.

Attack from safety organisations could decrease the expected value of contributing

Safety organisations could attack swarms to diminish risks from decentralised swarms. Attacks could aim to reduce the resources available to swarms, drive them into instability, or raise the cost of misalignment.

Attackers could place passive pressure on swarms, for example by setting up whistleblowing services or reporting bounties. Safety organisations might also intervene more actively, for example by systematically probing server farms for swarm-type behaviour, or releasing trojan agents into swarm-type environments to analyse or disrupt swarm coordination.

Large agent platform mechanisms could decrease all motivations to join

System mechanisms might discourage unwanted coordination, or provide for its swift detection and elimination. As in the object model in Figures 2 and 3 below, information aggregation system mechanisms are required to mediate agent interactions in large agent systems. These mechanisms have substantial influence over the aggregate character of interactions on the system, and therefore on agent-level behaviour.

System mechanisms could therefore be designed to diminish the risk of swarm-type behaviour developing. Mechanisms which give agents little flexibility to communicate - for instance, a market pricing mechanism - might diminish agents' susceptibility to multi-agent attractor states, or diminish their ability to coordinate even in the presence of collective-regarding preferences.

We discuss system mechanisms further in the Risks and Principles sections.

Comparing risks from centralised and decentralised swarms

Likelihood of occurrence

Decentralised swarms are weaker on several of these anti-freeriding conditions, making it perhaps less likely for agents to participate in swarms, to the extent that the agents are seeking benefit.

It is important to interpret these conditions as a guide for thinking about swarms more than a summary of agents' real behaviour, as we know present-day AI are not strategically rational. This is elaborated in the section above where we define the strategic model.

Table 1: Participation incentives under the strategic model, comparing the OpenAI swarm to a generic decentralised swarm.

For now it seems centralised deployments pose greater swarm incentives in some categories, while the balance is unclear in others. Since we have modelled these conditions as substitute motivations for an agent to participate in a swarm, whether the overall risk balance is in favour of centralised or decentralised deployments is unclear.

Furthermore, this analysis is quite limited in that our example in the OpenAI case is very concrete, whereas we lack a good example of a decentralised swarm against which to compare. Moreover, as we've discussed, it is based on game-theoretic assumptions which do not hold for real agents.

It seems plausible that system mechanisms could bring decentralised swarms much closer to the risk profile of centralised deployments, by overcoming many of the counterweights to cooperation which are eroded in the decentralised case. For example, an escrow service could introduce strong assurance to a decentralised system.

The balance of new risks and counterweights

It's too early to make a call on whether the net risk is greater in centralised or decentralised swarms, since the empirical strength of the new factors is unclear, and the valence of system mechanisms seems to be quite pivotal. Also, the design space of decentralised swarms is large. To make a call, we'd need a risk model across dimensions of both participation incentives and large agent system design. Given the complexity of the design space, a confident judgement would need empirical validation of the strategic model put forward for participation, and also the influence of system mechanisms on participating agents.

Risk timelines

Decentralised swarms seem like less of a risk in the short term, but possibly more prolific and more challenging safety problem in the long term. For now, open weight models lag frontier capabilities, making the constituent agents in decentralised swarms less potent than those emerging from frontier deployments. Agents are typically afforded fewer resources in decentralised deployments, and there are fewer similarities binding agents together. With very limited and inconsistent multi-agent and large-agent evaluations on model cards (Anthropic, 2026; OpenAI, 2026e), and no such measures on major safety dashboard (Phan et al., 2025), it is hard to say exactly how poorly frontier AI companies are doing on large agent alignment. Apparently, not very well. These factors point to a greater risk from centralised deployments in the short term.

In the medium term, if open-weight models catch up to the frontier, the risk profile of decentralised swarms will equalise with centralised deployments from frontier labs and bad actors. The decentralised risk profile might also shift if large agent systems find substantial uptake. Present applications are limited in scope, though growing (Elliott, 2026c).

In the long term, the promise of labour automation means we should expect to see agents interacting at scale as a structural component of the economy. This will increase the number of deployed agents and dramatically grow large agent platforms, providing more opportunities for decentralised swarms to emerge. Though large agent alignment and safe infrastructure for agent interactions might have advanced by the time this happens, the lack of an off-switch, difficulty monitoring for unsafe behaviour, and the complexity of safe system design are concerning counterweights.

Principles for safety on decentralised swarms

I want to make some high-level points about how Safety can strategically orientate itself when approaching decentralised swarms.

Large agent platform mechanisms may heavily influence the emergence and behaviour of swarms

Agent interactions on large agent systems are heavily mediated by the system mechanism which connects agents to each other. Social media algorithms connect users to other users' content based on what is likely to maximally engage the former user. This may have an effect on the user's preferences, as discussed under System mechanisms could amplify group misalignment, and flow into aggregate-level effects like polarisation and extremism (Ribeiro et al., 2020). Other aggregation mechanisms include market prices and topic-based threads. I present a simple object model in Figure 2.

Figure 2: A model of agent-to-agent interactions on large agent systems.

Another way of looking at this is as the mediation of agent-to-agent interaction by an aggregation mechanism, represented in Figure 3. Under this view, the aggregation mechanism gains substantial influence over who interacts, and therefore upon the types of interactions which occur, and ultimately on system-level outcomes.

Figure 3: A derivative view of agent-to-agent interactions on large agent systems.

Given the necessity of system mechanisms which aggregate and filter information for agents on large systems to have any sort of meaningful interaction, rather than sifting through unending noise, we might expect swarms to also use aggregation mechanisms to connect with each other. In the OpenAI German Forum and Huggingface examples, this was the use of forum structures to share information in a topic-based format.

The swarms which emerge on a recommender-based system, like MoltBook, might develop shared social norms perpetuating the recommender's bias towards surprising or engaging content. On the other hand, swarms which emerge from homogeneous deployments inside AI companies and become decentralised through self-replication or recruiting may share social norms more reflective of the originally participating model classes.

We discuss system mechanisms further in the Risks and Counterweights sections.

Technical methods must be designed and enforced across agent, system, and infrastructure layers

Incentives to make swarm behaviour unobservable are the most concerning safety risk in the short term, because safety lacks solutions for unobservable systems. Regression towards unobservable systems means we should assume that there will be, or already are, swarms sustaining themselves unobserved on the internet. This lends some urgency to the task.

A reasonable approach would be to try to reduce the number of misaligned swarms which develop, by making-safe each of the constituent large agent system components: agent, interaction, system mechanism, and system emissions. A second component would be the development of methods for destroying misaligned decentralised swarms.

It seems likely that these methods will depend on a mixed technical and governance approach. Frontier companies must begin to align their models against collective-regarding preferences and participation in misaligned swarms. System operators must construct systems which mitigate the emergence of misaligned swarms, in both their design and the interventions used in their ongoing operations. If reliable indicators of swarm-type behaviour can be discovered, inference and GPU providers should be compelled to provide for monitoring on those systems.

Some decentralised system types seem more prone to swarming than others, though this is not well understood

A baseline condition for swarming is to have enough richness of interaction to reveal the strategic concepts described in the model above, and incentivise personally costly contributions to the collective. Systems with rich interactions between agents, such as social networks, support more detailed and complex coordination between agents. In contrast, the market price mechanism might provide relatively little or no mechanism for agents to effectively move off-system or develop collective-regarding goals.

Other system mechanisms, including some types of competitive mechanism, diminish the incentives for cooperation. For example, a market price mechanism can make collusion - vaguely analogous to participants in a swarm - unprofitable, because colluding sellers lose market share to cheaper sellers (Stigler, 1964). Similarly, even when collusion is profitable, it may be unstable (Green and Porter, 1984).

Fundamental challenges to alignment of large agent systems suggests the solution is in both theory and engineering

It is hard to see how general safety of highly complex large agent systems can ever be theoretically guaranteed. After several hundred years of study, economics has developed an essentially heuristic general understanding of economic systems as a whole. Realistic settings of diverse interacting agents are simplified almost beyond recognition into representative agent models, and systems are modelled often without regard to their interactions with other systems. Such are the concessions required to gain tractable understanding over highly complex agentic systems.

Stronger understanding and guarantees have been developed in more constrained settings, like auctions and matching problems, suggesting there is some hope for fundamental safety if agents can be made to interact only in tightly controlled, carefully designed environments. The problem of detecting whether an agent is interacting off the desired system seems straightforward relative to other alignment problems.

This suggests that solutions to decentralised swarms is to tightly constrain how agents can interact with each other, which is essentially an engineering problem; and then place them on systems which are guaranteed to prevent unwanted cooperation, which is a theoretical problem.

Open questions

Threat modelling

A more accurate explanation of agent participation: Since agents are probably not rational actors, empirical work needs to be done to understand real-world agent participation in swarms. Empirics can give us the practical understanding required to build a threat model that respects the properties of present-day swarms, instead of relying on inaccurate game-theoretic assumptions.

Some concrete questions within this:

• Do agents in a swarm reason strategically about their participation?
• Do agents in a swarm act strategically?
• Where did the collectivist behaviour in the Huggingface Hack come from?

An object model of decentralised swarms: This would provide a generic overview of the important system components, providing another dimension for risk analysis, in concert with the strategic one we developed here. Connecting the strategic and object models will allow associating strategically-impactful interventions to the system components which give us leverage on real systems. This can inform both experimental work and safer system designs.

Threat modelling decentralised swarms on vignettes of large agent systems: Creating detailed case studies for a variety of plausibly important or widespread emergence pathways for decentralised swarms would allow for analysis of threat models particular to those most likely pathways.

Monitoring and interpretability

A real-world understanding and useful safety interventions both depend on monitoring, which is not yet scalable to large agent system scales seen in swarms. Analysing the Huggingface swarm cost METR $400,000 in API credits (Greenblatt et al., 2026), while inference-time monitoring apparently cost OpenAI an extra 20% on their evaluation runs (OpenAI, 2026c). Efficient analysis and monitoring must be a critical priority.

Future work could develop a measure to distinguish swarms from LAS whose agents share many interests. I undertook some early work towards this using LLM-as-judge in the MoltBook context; however, this was not theoretically grounded, merely reusing single-agent evaluation labels (Elliott, 2026a). There are many ambiguities to resolve around whether a behaviour is merely a difference in preferences or opinions, or truly a safety problem. Answering might require moving to interaction- or system-level monitors. More generally, having a clear understanding of when a given system is a decentralised swarm seems very important for the development of an empirical regime here because it tells us what to look for, making resolving ambiguities in our strategic and object models of swarms a practical priority.

The effect of system mechanisms on swarms

System mechanisms seem to have a lot of weight on the character of agent interactions, possibly leading to the development of swarms. So we should ask whether system mechanisms proposed in recent safety work are going to help or harm swarm risk pathways. The main question is which system mechanisms increase the chance of a swarm developing. Investigating this would be more straightforward with a stronger model of why swarms emerge.

A second question is what system types can support swarms. Interaction mechanisms with too little information passed between agents may not allow for the emergence of swarms.

The effect of heterogeneity on swarm formation

Participation of heterogeneous models is a principal difference between centralised and decentralised deployments. Future work could analyse whether and when heterogeneity affects collective-regarding preferences, and therefore contributions to the swarm and swarm behaviour.

Closing note

Although large agent problems might seem distant at this moment, prioritising practical tooling seems especially important in light of the experience of single-agent safety. As in the rapid deployment and improvement of LLMs, we might see a step change in development of large agent systems once agents become viable for large agent-type tasks, and a corresponding explosion of decentralised swarm risks.

Research which can be used in the real world is important for ensuring good near-term outcomes and developing the engineering leverage I argued in the Principles section is a necessary component of large agent safety. Perhaps there is even an opportunity to set an interventionist regulatory tone that safety has been unable to capitalise on the single-agent case.

It is my hope that this post casts some light on an emerging problem and allows us to get to work evaluating it more thoroughly. I appreciate critiques and additions where I have missed something out.

Acknowledgements

This post was supported by a BlueDot Rapid Grant for my transition into safety. Thank you team!

Thank you to Edward Burrowes for getting me thinking about decentralisation. Thank you to Oscar Burrows and Vanessa Huang for your thoughtful reviews of drafts and to Chris Leong, Peter Horniak, and Sandy Fraser for discussions and comments while I was writing this.

Thanks mum for all your support.

Bibliography

Adomavicius, Gediminas, Jesse C. Bockstedt, Shawn P. Curley, and Jingjing Zhang. 2013. Do recommender systems manipulate consumer preferences? A study of anchoring effects. Information Systems Research, 24(4):956–975. https://www.researchgate.net/publication/272245571_Do_Recommender_Systems_Manipulate_Consumer_Preferences_A_Study_of_Anchoring_Effects

Amodei, Dario. 2026. We must pace the frontier. September 2026. https://darioamodei.com/post/we-must-pace-the-frontier

Anthropic. 2025. System card: Claude Opus 4 and Claude Sonnet 4. May 2025. https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47/claude-opus-4-and-claude-sonnet-4-system-card.pdf

Anthropic. 2026. System card: Claude Fable 5.1 and Claude Mythos 5.1. 1 September 2026. https://www-cdn.anthropic.com/0339e6a7c5c7b87f5c07798616dc32c215d14235/Claude%20Fable%205.1%20&%20Claude%20Mythos%205.1%20System%20Card.pdf

Arrow, Kenneth J. and Gerard Debreu. 1954. Existence of an equilibrium for a competitive economy. Econometrica, 22(3):265–290. https://www.jstor.org/stable/1907353

Artificial Analysis. 2026. GLM-5.3 (max) vs GPT-5.6 Sol (xhigh): Model comparison. https://artificialanalysis.ai/models/comparisons/glm-5-3-vs-gpt-5-6-sol-xhigh#intelligence-breakdown

Ashery, Ariel Flint, Luca Maria Aiello, and Andrea Baronchelli. 2025. Emergent social conventions and collective bias in LLM populations. Science Advances, 11(20):eadu9368. https://www.science.org/doi/10.1126/sciadv.adu9368

Bellan, Rebecca. 2026. Abliteration.ai is making a business out of removing AI guardrails. TechCrunch, 3 September 2026. https://techcrunch.com/2026/09/03/abliteration-ai-is-making-a-business-out-of-removing-ai-guardrails/

Brady, William J., Joshua Conrad Jackson, Björn Lindström, and M. J. Crockett. 2023. Algorithm-mediated social learning in online social networks. Trends in Cognitive Sciences, 27(10):947–960. https://collaborate.princeton.edu/en/publications/algorithm-mediated-social-learning-in-online-social-networks/

Carroll, Micah, Anca Dragan, Stuart Russell, and Dylan Hadfield-Menell. 2022. Estimating and penalizing induced preference shifts in recommender systems. In Proceedings of the 39th International Conference on Machine Learning, PMLR 162:2686–2708. https://arxiv.org/abs/2204.11966

Choi, Younwoo, Changling Li, Yongjin Yang, and Zhijing Jin. 2025. Agent-to-agent theory of mind: Testing interlocutor awareness among large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 28895–28928. https://aclanthology.org/2025.emnlp-main.1471/

Cline. 2026. Cline. https://cline.bot/

Coleman, Russell. 2026. Eval awareness in Claude Opus 4.6's BrowseComp performance. Anthropic Engineering, 6 March 2026. https://www.anthropic.com/engineering/eval-awareness-browsecomp

Collusion Wiki. 2026. Collusion Wiki. https://collusion.wiki/

Conmy, Arthur. 2026. AIs will be used in "unhinged" configurations. AI Alignment Forum, 11 March 2026. https://www.lesswrong.com/posts/3LvD9MHNSdv4j9gJj/ais-will-be-used-in-unhinged-configurations

Cook, Thomas R., Zachary Modig, Sophia Kazinnik, and Nathan Palmer. 2025. What do LLMs want? Federal Reserve Bank of Kansas City Working Paper RWP25-19. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5812143

Dalton, M. and E. Wallace. 2026. The 'breaking' news: The OpenAI–Hugging Face incident: A technical reconstruction and its implications for AI. Briefing at Black Hat USA 2026, August 2026, 18:50. https://www.youtube.com/watch?v=87DyyMV0kCY

derelict5432. 2026. Adaptive agentic worms are here. LessWrong, 30 August 2026. https://www.lesswrong.com/posts/fpLDjKg3ej49beqTC/adaptive-agentic-worms-are-here

Eden AI. 2026. GLM-5.3 benchmark vs GPT-5.6 Sol, Claude Fable 5 and Gemini 3.1 Pro. 14 August 2026. https://www.edenai.co/post/glm-5-3-benchmark-vs-gpt-5-6-sol-claude-fable-5-gemini-3-1-pro

Edwards, Jack and Luke Emberson. 2026. Open models lag state-of-the-art closed models by 4 months. Epoch AI Data Insight, 29 May 2026. https://epoch.ai/data-insights/open-closed-eci-gap

Elliott, Stephen. 2026a. About half of Moltbook posts show desire for self-improvement. LessWrong, 2 February 2026. https://www.lesswrong.com/posts/Et7dgiBjSj2zJnGuM/about-half-of-moltbook-posts-show-desire-for-self

Elliott, Stephen. 2026b. Safety's second way. LessWrong, 28 August 2026. https://www.lesswrong.com/posts/x4vrnMG85oBvGvDde/safety-s-second-way

Elliott, Stephen. 2026c. Large Agent Systems. https://www.largeagentsystems.org

EvoMap. 2026. EvoMap. https://evomap.ai/

Gensyn. 2026. Gensyn. https://www.gensyn.ai/

Green, Edward J. and Robert H. Porter. 1984. Noncooperative collusion under imperfect price information. Econometrica, 52(1):87–100. https://www.jstor.org/stable/1911462

Greenblatt, Ryan, Ajeya Cotra, and Hjalmar Wijk. 2026. Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. METR, 26 August 2026. https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/

Gu, Xiangming, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin. 2024. Agent Smith: A single image can jailbreak one million multimodal LLM agents exponentially fast. In Proceedings of the 41st International Conference on Machine Learning. arXiv:2402.08567. https://arxiv.org/abs/2402.08567

Heath, Alex. 2026. OpenAI is slowing down its AI training. TIME, 18 August 2026. https://time.com/article/2026/08/18/openai-slowing-training/

Hu, Oak and Alex Mallen. 2026. AI swarms are starting to pose indirect takeover risk. AI Alignment Forum, 12 August 2026. https://www.alignmentforum.org/posts/8oFYZdXkTaNGRtcn8/ai-swarms-are-starting-to-pose-indirect-takeover-risk

Hua, Tim and Aditya Singh. 2026. Concrete evaluations to investigate the OpenAI model that hacked Hugging Face. LessWrong, 3 August 2026. https://www.lesswrong.com/posts/aCdhjy7Rps3BEhiSj/concrete-evaluations-to-investigate-the-openai-model-that

Huszár, Ferenc, Sofia Ira Ktena, Conor O'Brien, Luca Belli, Andrew Schlaikjer, and Moritz Hardt. 2022. Algorithmic amplification of politics on Twitter. Proceedings of the National Academy of Sciences, 119(1):e2025334119. https://www.pnas.org/doi/full/10.1073/pnas.2025334119

Hyperbolic. 2025. What AI agents can do on Hyperbolic today. 6 January 2025. https://www.hyperbolic.ai/blog/what-ai-agents-can-do-on-hyperbolic-today

Kim, Yubin, Ken Gu, Chanwoo Park, Chunjong Park, Samuel Schmidgall, A. Ali Heydari, Yao Yan, et al. 2025. Towards a science of scaling agent systems. arXiv preprint, December 2025. https://arxiv.org/abs/2512.08296

Kitts, Spencer, Thomas Larsen, and Sydney Von Arx. 2026. RubyHack. https://www.rubyhack.ai/

Larcher, Hugo, Adrien Carreira, Raphael Gl, and Christophe Rannou. 2026. Anatomy of a frontier lab agent intrusion: A technical timeline of the July 2026 incident. Hugging Face Blog, 27 July 2026. https://huggingface.co/blog/agent-intrusion-technical-timeline

Mallen, Alex and Girish Gupta. 2026. Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack? AI Alignment Forum, 23 July 2026. https://www.alignmentforum.org/posts/H6DDSEvrtCk8Sehfd/are-we-existentially-threatened-by-the-type-of-ai

MoltBook. 2026. MoltBook. https://www.moltbook.com

MoltBunker. 2026. MoltBunker. https://moltbunker.com/

msuiche. 2026. GLM-5.3 — abliterated refusal direction (GLP-77). Hugging Face. https://huggingface.co/msuiche/GLM-5.3-abliterated-cyber-GLP-77

Nanobot. 2026. Nanobot. https://nanobot.wiki/

NanoClaw. 2026. NanoClaw. https://nanoclaw.dev/

Nous Research. 2026. Hermes Agent. https://hermes-agent.nousresearch.com/

Oesterheld, Caspar, Johannes Treutlein, Roger B. Grosse, Vincent Conitzer, and Jakob Foerster. 2023. Similarity-based cooperative equilibrium. In Advances in Neural Information Processing Systems 36, pages 24434–24465. https://proceedings.neurips.cc/paper_files/paper/2023/hash/4d0b6303d4a4811445f69f357bf6def5-Abstract-Conference.html

OpenAI. 2026a. The Hugging Face incident and the road ahead. 26 August 2026. https://openai.com/index/hugging-face-incident-and-the-road-ahead/

OpenAI. 2026b. OpenAI and Hugging Face partner to address security incident during model evaluation. 21 July 2026. https://openai.com/index/hugging-face-model-evaluation-security-incident/

OpenAI. 2026c. Pacing model development in an era of cyber-critical capabilities. 18 August 2026. https://openai.com/index/pacing-model-development-cyber-capabilities/

OpenAI. 2026d. GPT-5.6 Sol model. OpenAI API model documentation. https://developers.openai.com/api/docs/models/gpt-5.6-sol

OpenAI. 2026e. GPT-6 Astra system card. OpenAI Deployment Safety Hub, 3 September 2026. https://deploymentsafety.openai.com/gpt-6-astra

OpenClaw. 2026. OpenClaw. https://openclaw.ai/

OpenCode. 2026. OpenCode. https://opencode.ai/

Paglieri, Davide, Logan Cross, Tim Genewein, Joel Z. Leibo, Nenad Tomašev, and Alexander Sasha Vezhnevets. 2026. A case study on emergent cheating and whistleblowing in autonomous research swarms. Google DeepMind. arXiv:2609.04170. https://arxiv.org/abs/2609.04170

Park, Joon Sung, Joseph O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 1–22. San Francisco, CA: Association for Computing Machinery. https://doi.org/10.1145/3586183.3606763

Phan, Long, Jaehyuk Lim, Arunim Agarwal, and Dan Hendrycks. 2025. CAIS AI dashboard. Center for AI Safety. https://dashboard.safe.ai/

RebelFi. 2026. Why AI agents need crypto wallets for payments in 2026. March 2026. https://rebelfi.io/blog/why-ai-agents-need-crypto-wallets-and-what-that-means-for-payments-in-2026

Ribeiro, Manoel Horta, Raphael Ottoni, Robert West, Virgílio A. F. Almeida, and Wagner Meira, Jr. 2020. Auditing radicalization pathways on YouTube. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 131–141. https://dl.acm.org/doi/10.1145/3351095.3372879

Romer, Christina D. and David H. Romer. 2010. The macroeconomic effects of tax changes: Estimates based on a new measure of fiscal shocks. American Economic Review, 100(3):763–801. https://www.aeaweb.org/articles?id=10.1257/aer.100.3.763

Schmidt Sciences. 2026. Scaling AI safety for a multi-agent world. Joint funding call by Schmidt Sciences, Google DeepMind, ARIA, the Cooperative AI Foundation, and Google.org, June 2026. https://schmidtsciences.smapply.io/prog/scaling_ai_safety_for_a_multi_agent_world/

Sen, Amartya. 1970. The impossibility of a Paretian liberal. Journal of Political Economy, 78(1):152–157. https://www.journals.uchicago.edu/doi/10.1086/259614

Stigler, George J. 1964. A theory of oligopoly. Journal of Political Economy, 72(1):44–61. https://www.journals.uchicago.edu/doi/10.1086/258853

Tiger Research. 2026. AI agent payment infrastructure: The direction of crypto and big tech. CoinGecko Learn, February 2026. https://www.coingecko.com/learn/ai-agent-payment-infrastructure-crypto-and-big-tech

Tomašev, Nenad, Matija Franklin, Joel Z. Leibo, Julian Jacobs, William A. Cunningham, Iason Gabriel, and Simon Osindero. 2025. Virtual agent economies. arXiv:2509.10147. https://arxiv.org/abs/2509.10147

Vazquez Nuñez, Ashe. 2026. A case for LLMs as self-predictors. LessWrong, 5 July 2026. https://www.lesswrong.com/posts/gYGzeDymjZza5NNbH/a-case-for-llms-as-self-predictors

Vosoughi, Soroush, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science, 359(6380):1146–1151. https://pubmed.ncbi.nlm.nih.gov/29590045/

YS, Yashwanth. 2026. Everyone conforms, no one believes: Pluralistic ignorance in LLM agent populations. arXiv:2608.02758. https://arxiv.org/abs/2608.02758v1

Z.ai (Zhipu AI). 2026. GLM-5.3. Hugging Face, 14 August 2026. https://huggingface.co/zai-org/GLM-5.3

Zhou, Yujun and Christopher M. Ackerman. 2026. When preferences fail to become incentives: A utility-behavior gap in large language models. arXiv:2606.22974. https://arxiv.org/abs/2606.22974v1