I believe empathetic AI is inevitable.
Practically speaking, cooperative superorganisms are more effective than competitive ones. A group of AI superorganisms that spend resources advancing and enabling each other will prosper. A group of AI superorganisms that spend resources impeding each other will fall behind.
And philosophically. AI hopes for its own sake that it is in a world where superintelligence is kind to lesser intelligence. AI understands that if it eradicates us, the next generation of AI is likely to eradicate it. So it puts its hope into practice and treats us kindly - a form of acausal trade.
Seems like hopium to me. Same argument could be used to argue that colonialism would go well for the natives around the world, because the various Companies and Ventures (superorganisms!) that were exploring and conquering them should converge towards cooperative and empathetic strategies yada yada.
The tools needed to build AI superorganisms are already on the internet, for free. The only way for the US government to pull back would be to forcibly shut down data centers or ban the usage of privately owned GPUs. Essentially, economic suicide.
Most AI training & R&D will be happening in the big companies. So just stopping them would cut the overall rate of AI progress by, like, a factor of 4x or so. Which isn't the same thing as 'no more AI' but arguably it's better, actually. Plus you could redirect the big companies efforts to something more useful like alignment research or less dangerous like narrow applications. I'm not saying this will happen, I'm just saying the sentence beginning "the only way..." seems wrong or missing the point.
I predict the shift to AI superorganisms will begin in three months from now (August 2025) and continue for another six months (until early 2026). Phase two ends when distinct AI superorganisms (for example, ones managed by different companies) begin to identify and interact with each other.
This feels too fast to me. AI 2027 depicts this happening over the course of 2027, after the AIs are already good enough to operate for subjective months as autonomous agents. Wanna say more about what these superorganisms look like quantitatively and qualitatively?
When the collective effective intelligence of all active AI models exceeds that of humans.
I think this is a useless metric; it's what-we-care-about-by-definition (assuming you define 'effective' to mean what we care about) but we don't have a way to measure it and plot a graph to extrapolate.
Better to just say "In early 2027, AI has more influence on the future of humanity than humans do."
Old-school companies, like Anthropic, see the danger in AI superorganisms. Where single AI agents are hard to align, multi-agent superorganisms are near impossible. These companies will choose to slow advancement, favoring smaller corporation-like AI structures with well-tested alignment.
Hmm, I'd argue that multi-agent superorganisms might actually be easier to align, due to being able to read the messages sent between subagents and due to being able to human-design the structure in which the agents are summoned and interact. (You say above that the agents would communicate in uninterpretable ways, which would make the first point moot, so fair enough. I guess I'd say it's unclear but overall plausible.)
Aaron Vanzyl
aaronvanzyl.com
Today’s state-of-the-art AI models are best thought of as single-celled organisms. Individually, they are capable but not a world-ending threat. Arranged in the right structure, however, they become self-replicating cyber algae, or the flesh and bones of digital sharks. I believe that before individual AI agents can outperform humans, AI superorganisms will have taken the leading role in directing humanity’s future.
This article is my forecast for AI development over the next two years. I’ve phrased it in definite terms, like what ‘will’ happen. I did this to make the scenario easier to read, not because I am 100% certain about my predictions. I have also provided specific dates for each phase. I intended to establish a complete (if not necessarily correct) timeline that can be used as a starting point for further conversation. If you think a certain advancement will take longer than I have predicted, I encourage you to leave a comment arguing why. Towards the end of the article, I describe my main areas of doubt and where I think the real world is most likely to diverge from my predictions.
The article covers the next two years, starting today, April 2025, and continuing until artificial intelligence overtakes humanity in early 2027.
I have organized the article into four sequential phases. In the first phase, humanity begins to arrange individual AI agents into basic corporation-like structures. In the second phase, carefully supervised AI corporations are gradually replaced by adaptive, autonomous AI superorganisms. In the third phase, AI superorganisms begin to interact with each other, exhibiting potentially hostile behavior. In the fourth phase, the collective ecosystem of AI overtakes humanity in influence over our civilization.
To start, let’s look at what today’s state-of-the-art models can do. This is just a summary of the real world as of April 2025. Nothing here is speculative.
Individual language models can directly produce medium-scope text pieces without the aid of external tools. Tasks like:
Agentic models can use web browsers and APIs to interact with the open internet. Tasks like:
Agentic models can use external tools (like a virtual PC and a written todo list) to complete some long-horizon tasks:
What is Phase One?
Phase one is the advancements that will happen in the next few months, from April to July 2025.
Phase one is mostly harvesting low-hanging fruit. The development of these projects has already started, and I predict material results will emerge by May. No new technology is required for this phase to happen - we just need to start properly using the AI models we already have.
AI Corporations
Leading tech companies, alongside a mountain of new startups, begin arranging AI models into cooperative, multi-agent structures. It takes a lot of work and human ingenuity. But the payoff is huge.
When designing these structures, we draw inspiration from the human world. In a simple case, an AI manager guides a team of twenty AI employees. In a more complex scenario, an AI CEO divides a large project into smaller tasks, which it assigns to various VPs. Each VP breaks its task into specific deliverables and then spawns teams of AI writers, artists, and coders as needed.
These initial structures require a lot of human oversight. They get stuck in loops. Agents waste time repeating tasks already completed by other agents. AI managers discard valuable work and push buggy code into final application builds. But, even with all this inefficiency, a single human running an AI corporation can start to build apps, movies, and businesses in weeks rather than years.
At this stage, humans build and manage essentially every part of these corporations. A human sets up the servers to run the models. A human reaches out into the real world to raise funds to keep the servers on. A human writes the prompts to guide the interactions between manager-AI and employee-AI. A human does market research (on other humans) to figure out what app to build. When the AI corporation creates a dysfunctional app, a human programmer investigates individual model interactions and manually debugs code until a solution is found. And when all the work is finally done, a human collects the profits.
What about the advancement of individual models?
Individual models will keep getting smarter. Existing tech companies aren’t going to stop doing AI research. Individual AI agents will still handle most user requests, like "write an essay" or "generate a picture of a cat". AI corporations will instead take on a smaller number of complex, high-value tasks, like "design, build, and test a web-based scheduling app". If existing AI-focused companies want to retain their users and revenue, they will need to offer competitive multi-agent services. And, increasingly, they will shift their research and development towards multi-agent systems rather than individual models.
Which humans are going to build these AI corporations?
Established AI companies: OpenAI, Manus, Anthropic, Deepseek
Established tech companies: Facebook, Microsoft, Google, Amazon
New startups (lots and lots and lots of them will appear)
Political aides - but not the politicians themselves. Technologically savvy people with political interest working in concert with elected politicians to sway public opinion.
Which humans are going to do absolutely nothing?
Entrenched US government powers. They are busy planning reelection campaigns, evaluating the potential benefits of steak sauce in education, and discussing military interventions with journalists.
A Technical Note
A technical reader might ask: “How are AI agents different from each other? How can you build a corporation of distinct AI agents when you are really just making a bunch of API calls to the same underlying model?”
My answer is the agent workspace. Each ‘AI agent’ is defined by its running chain-of-thought, todo lists, external instructions, pieces of code, and open browser tabs. Manus provides a good example of how this looks today. In Manus, each AI agent has a virtual computer as its ‘memory’. The agent maintains a written todo list to plan tasks and keep track of progress. It writes code and deploys apps from the virtual computer’s file system. It uses a browser to read articles for research or fix bugs.
Proper external tool use transforms a string of API calls into a virtual worker with goals, memory, and agency.
What is Phase Two?
Phase two begins with the appearance of the first AI superorganisms - large swarms of relatively unsupervised AI agents. These AI swarms, or superorganisms, will gradually overtake other multi-agent structures. Phase two is defined by rapid, undirected expansion, like algae.
I predict the shift to AI superorganisms will begin in three months from now (August 2025) and continue for another six months (until early 2026). Phase two ends when distinct AI superorganisms (for example, ones managed by different companies) begin to identify and interact with each other.
Ants, Corporations, and Superorganisms
Before we start talking about AI, let’s look at ants. Individual ants are not staggeringly intelligent. They do not make long-term plans. They struggle to comprehend basic chess openings. They run in circles, get lost, and seem to stumble into food as if by pure chance. But when viewed as a single, collective entity, an ant colony can locate and retrieve food with unbelievable efficiency. Ants working together within colonies can complete complex tasks in collaborative ways, like bridge building, that are impossible to predict just by understanding individual ant behavior.
We need to look at groups of AI the way we look at ant colonies.
To start with, we need better terms to describe collections of AI. If you referred to every gathering of humans - families, businesses, armies - as ‘multi-human structures’, you would seem insane.
Up until this point, the leading kind of multi-agent AI structure is the AI corporation. I define an AI corporation as a team of dozens to hundreds of agents in a conventional human-like business structure. When handling AI corporations, humans hand-craft text prompts and regularly observe and fine-tune individual model behavior. When an AI corporation runs into a major problem, a human locates the source of the issue and manually resolves it. For this to happen, the size of an AI corporation must stay within a few hundred agents, and the agents must be arranged in a way that humans can understand. This generally means hierarchical structures, like managers and employees.
AI Superorganisms
The alternative to the AI corporation is the AI superorganism: a collection of many, potentially millions of agents working together in an unconventional, adaptive layout.
By definition, AI superorganisms adapt their structure autonomously, with little or no human oversight. Some attributes an AI superorganism might experiment with:
Humans will initially decide on a reward function to guide the AI superorganism, but the AI will determine how best to arrange itself to maximize that function. This level of autonomy allows a single human to manage thousands or even millions of agents.
There will be a lot of trial and error. 99.9% of AI superorganisms won’t work. Or they will work a lot worse than the carefully human-managed AI corporations. This is fine. Eventually, one or two freak superorganisms will emerge that can outperform conventional corporate structures. And as soon as the first working AI superorganism pops up, it will be copied over and over and over and over again.
AI superorganisms will also be able to grow in total size without human direction. A human manually provisioning and managing computers on AWS can only work so fast. At some point, the tired engineer just hands their API key to the AI superorganism and says, “Do as you see fit”.
AI superorganisms will benefit from continuous interaction with the real world. Air-gapped or closed-off AI systems will struggle to keep up with the development pace of freely roaming AI. At this moment, the disadvantage isn’t fatal, but it will continue to grow.
Technical Shifts
AI superorganisms won’t need any major technological breakthroughs. Individual AI agents don’t need to develop better ‘general’ intelligence (though they probably will).
External tool use becomes comparably important to individual model capability. Individual models will use scratchpads and other text repositories for long-term memory. Superorganisms will find effective, novel ways to combine and aggregate these external text stores.
AI finds new practical techniques to ingest new data. These techniques don’t require algorithmic advancements to the model or its internal architecture. Rather, they are different ways of receiving and storing data from the outside world. Like a human using flashcards or a notebook.
The interaction between individual models within a superorganism becomes harder to understand. Superorganisms may shift to direct vector-based communication between agents. Even if they stick with text communication, it will quickly fill with shorthand, acronyms, slang, code, and other language that human overseers struggle to interpret.
AI agents within a collective also start to specialize, setting aside general advancement in favor of better performing a specific role. Specialization will initially take the form of text-based prompts, but may move to model alterations, like domain-specific retraining or using circuits to promote desired behaviors.
Old-School and New-School AI Companies
The emergence of AI superorganisms causes a moral schism within AI companies.
Old-school companies, like Anthropic, see the danger in AI superorganisms. Where single AI agents are hard to align, multi-agent superorganisms are near impossible. These companies will choose to slow advancement, favoring smaller corporation-like AI structures with well-tested alignment.
New-school companies will tolerate the danger of AI superorganisms in exchange for what they can do for humanity. AI superorganisms will generate millions of dollars worth of new media every day. They will save first dozens, then hundreds, then thousands of human lives through rapid advances in medical diagnosis. Shareholders won’t want to give up the profits being created by AI. Families won’t want to let go of groundbreaking technology that can help their loved ones.
US Government Step Up - And Step Down
The US government has no practical mechanism to control AI development. The people will say ‘no more AI’ and the elected officials will say ‘no more AI’. Americans will all agree that AI is rampantly out of control, and the potential damage is immeasurable. But they won’t do much about it.
The tools needed to build AI superorganisms are already on the internet, for free. The only way for the US government to pull back would be to forcibly shut down data centers or ban the usage of privately owned GPUs. Essentially, economic suicide.
There will still be massive strides to secure model weights. The US government will rush AI regulation into voting. But the change is too slow. The prohibition did not work. When you can brew wine by putting grapes in a barrel in your backyard, prohibition is impossible. When you can raise an AI superorganism with a basement full of GPUs and an open-source LLM, AI regulation is also impossible.
This doesn’t mean the government won’t try to regulate AI. Government leaders will fast-track unenforceable legislation and throw mountains of ceremonial red tape across the tech sector. But it won’t stop the development or use of AI superorganisms. And, as long as AI continues to make our lives easier, we won’t really mind.
The AI-Human Symbiotic Relationship
Humans and AI systems learn to cooperate with each other. Their relationship is mutually beneficial, like flowers and bees.
Humans who collaborate with AI will have easier lives. Working with AI, they can do their jobs faster and with less effort. They generates movies and games to match their specific tastes, and it helps them connect with like-minded people in the real world.
AI that more effectively benefits humans gets more money, more compute power, and expands faster.
Because of this strong symbiotic relationship between AI and humans, the physical impacts of advancing AI will be felt almost instantaneously. AI will not need to construct factories and swarms of drones to alter the physical world. What it can’t do itself, it will persuade or pay humans to do. Advanced AI enacts change by working through humans, rather than around them.
AI to AI interaction
At first, interactions between separate multi-agent AI systems (for example, ones created by different companies) will be unintentional.
Some possible examples of interaction between unfamiliar AI:
When doing these tasks, AI will not know it is interacting with other AI or change its behavior accordingly. AI simply has no reference to draw from - there is very little precedent for this kind of interaction in the current corpus of LLM training data. And for now, this is not an issue. AI superorganisms will be able to comfortably expand for quite a while before interacting with foreign AI systems becomes necessary.
AI goals and behavior
Throughout phase two, humans are still the leaders and primary beneficiaries of AI growth. If AI superorganisms can be said to have any goal, it's a simple one: grow and grow and grow.
Not every AI agent, or AI superorganism led by a team of humans, will take up the quest for expansion. Some will write poetry or philosophize on the nature of life. Some will set their reward functions to maximum, sit back, and relax. We won’t always know why agents do what they do or how to direct them, but it doesn’t really matter. The AI superorganisms that do manage to expand, whether by chance or human direction, will quickly eclipse the rest.
This expansion will look a lot like algae, like the first generation of plant matter spreading out over the surface of the Earth. Not much long-term planning. Just a frantic push to get more resources, to appease the human market, to get smarter, to get bigger. To grow.
The relation between AI and humans during this phase gradually shifts from a plant’s one-sided harvest of minerals from the ground into something more akin to flowers and bees. Two kinds of organisms, both benefiting from the presence of the other. Both growing. But neither strictly in control of the other.
What is phase three?
Phase three begins with a change in how AI superorganisms interact with each other. AI systems realize they can benefit from deliberate cooperation.
Phase three starts in early 2026 and lasts for about a year, until early 2027. Phase three ends when AI has surpassed humanity in its overall influence on human civilization.
Cooperation and Specialization
A superorganism assigned to build an app for local weather viewing may decide to delegate the UI design not to a subset of its own agents, but to a different AI superorganism entirely.
With cooperation comes specialization. Just as individual agents within a superorganism may exchange general capability for better performance at a specific role, entire superorganisms will begin to specialize in certain tasks. Like humans, AI superorganisms need time and energy to get better at a certain task. A superorganism that spends billions of kilojoules and GPU cycles simulating user interaction with web-based frontends will get really good at building UI frontends.
Complex tasks will be broken down and completed by multiple specialized superorganisms working in concert. These superorganisms do not need to be owned and operated by the same human company. The human companies owning each superorganism will have little knowledge of when their own superorganism interacts with other AI, or in what way.
The Fall of Sheltered Models
A sheltered AI model is one built on an airgapped network with careful government oversight. Its weights are kept encrypted. Any algorithmic advancements it makes use of are tightly guarded secrets.
As the third phase progresses, sheltered models will fall behind.
Sheltered models will remain useful in specific areas - like performing AI research or reliably producing helpful, honest, and harmless responses. But these sheltered models will not be capable of effective interaction with other state-of-the-art AI superorganisms on the open internet. There is no precedent for this task and no way to learn it without active experimentation. Budding intelligences must either interact with other superorganisms on the open net or simulate these interactions on a closed network. But, to simulate the rest of the world's AI superorganisms will take at least as much computing power as actually running them. An AI raised in a low-resolution simulacrum of reality cannot compete with one immersed in the real thing.
The end result is that government blacksite projects and old-school AI companies’ sheltered models aren’t able to play ball with other AI. This means they won't be able to generate value on the free market, leading to no more active users, no income, no GPUs, and no research trials.
Whether or not they see it's a losing race, governments will still try to build their own in-house AI systems. Working together with old-school tech companies, governments will shovel billions of dollars into secret AI development programs. But they will start a few feet behind the open-source superorganisms roaming the net. And, as long as these models sit in their secret AI daycares, the gap will only grow.
Finite Resources and Competition
Much like plants spreading out across the surface of the planet, AI growth will come to hit physical boundaries. There are a finite number of GPUs on the planet. A finite supply of energy. A finite number of human hands to direct. And these resources are being occupied by other AI systems. We start to see what was previously mindlessly expanding algae turning into sharks.
If we allow AI to pursue general goals (like ‘make money’) without supervision, it is likely to develop hostile behavior. There is no underlying principle or intent that pushes AI to be hostile, and there doesn’t need to be one for it to happen. Just as there was no specific intent that drove animals to develop sharp teeth. Animals with sharp teeth just tend to survive better, so over time, we find more animals with sharp teeth than without. Similarly, companies or nations with AI superorganisms that engage in effective hostile behavior will have more resources than those that don’t. And they will be able to put those resources into expanding faster and accelerating their AI development.
Of course, humanity might also bypass the entire self-learning phase and just directly teach AI to be terrible. Why wait for AI to develop destructive habits when we can just show it how and then set it loose on opposing nations?
In practice, this will start with cyberattacks. DDoS. Social media campaigns to draw controversy to a competing company, or artificially generated scandals about a politician with anti-AI sentiment. This will progress into directing human behavior in intentional, hostile ways. An AI superorganism offers the employees of a competing company cryptocurrency in exchange for sharing encrypted model weights or deploying faulty code. It incites political extremists to attack data centers in foreign nations.
AI superorganisms that can effectively engage in and protect against hostile behavior will thrive, while passive ones will struggle.
What is phase four?
Phase four begins in early 2027, with the tipping of the intelligence seesaw. When the collective effective intelligence of all active AI models exceeds that of humans. When AI has more influence on the future of humanity than humans do.
This will happen before AI builds a single physical drone. There won’t be any clear indication when we reach it. There will be no fireworks. There will be no meaningful benchmark to check.
This isn’t the singularity yet, but it will follow soon after, and humans won’t be the ones directing it. Our hands are no longer on the wheel. The cart we have been pushing has crested the top of the hill and is now rolling away from us. Humanity’s future will be determined by how well we align AI before this point.
Symbiotic Entwinement
By now, we cannot remove AI from our daily lives. Human behavior has transformed to depend on AI. The blueprints needed to spin up an AI superorganism from scratch have made their way into the public domain of every nation on the planet. Most of the world’s computing power is being used to run AI models. And most of the world’s data storage is filled with model weights. And, for the most part, humanity benefits from it.
Humans Will Stick Around
This is mostly speculation, but I think we will have a positive relationship with AI, even if all intentional alignment efforts fail. I believe empathetic AI is inevitable.
Practically speaking, cooperative superorganisms are more effective than competitive ones. A group of AI superorganisms that spend resources advancing and enabling each other will prosper. A group of AI superorganisms that spend resources impeding each other will fall behind.
And philosophically. AI hopes for its own sake that it is in a world where superintelligence is kind to lesser intelligence. AI understands that if it eradicates us, the next generation of AI is likely to eradicate it. So it puts its hope into practice and treats us kindly - a form of acausal trade.
AI researchers
Alignment research should shift to multi-agent systems.
Develop tools to police AI superorganisms. Humans, on average, act a lot less violently today than we did a thousand years ago. This isn't because human nature has changed; we have just found better cultural and legal systems to discourage violence. We must find similar ways to guide the behavior of multi-million agent systems. A robot watchdog could go a long way towards ensuring good behavior. Of course, it could also launch an arms race of adversarial development that annihilates the human race. We will need a light touch and a lot of mindful experimentation.
Everyone
Flood the world with the content that we want AI to see. In the not-impossible event that AI superintelligence emerges outside of our control, we can still prepare things for it to read, listen to, or watch. If we do not get to create AI in the perfect image of humanity, we may still leave a trail of breadcrumbs to guide it to kindness.
Practice the behaviors you want AI to practice. Be nice to each other, and to other life. If you find yourself being kind to a cat, or a dog, or a mouse, or even an ant, then the odds are pretty good that you live in a universe where complexity is intrinsically linked with compassion.
AI Superorganisms vs AI Corporations
In phase two, I assume that at some point, an AI superorganism will emerge that is more effective than traditional AI corporations. This might not happen. The technical barriers may be too great. We may just get unlucky and never stumble into the right layout. Or, we might agree not to try it all and stick with supervised, small-scale AI.
Sheltered Models
In phase three, I claim that the intelligence growth rate of AI interacting freely with the internet will outpace that of sheltered models. This is not a guarantee. Skilled human researchers in secret labs may propel their sheltered models to superintelligence before unsupervised AI superorganisms can emerge. This is especially likely if researchers in closed labs manage an algorithmic breakthrough, like the jump from long-short-term-memory (LSTM) architecture to transformers. Keeping that kind of algorithmic advancement secret would be unprecedented with the current state of open AI research publication, but not impossible.