Training models to coordinate across multiple agents gives us a way to productively spend substantially more compute during post-training than current single-agent RL setups.
But that’s one data point. We don’t know how long it would take a single agent to solve Navier-Stokes, because we haven’t done that experiment yet. Maybe we will, but that’s also only one data point.
If we want to do a thorough ablation, the experiments are just too expensive at that scale. So we have to do some kind of methodical science about what happens when you go to 64, 128, 256 or something and get a sense of the behavior. But it’s going to be very hard to push that all the way to 10,000 and know for sure what the benefit was that we actually got from using 10,000 agents versus 1,000.
There’s one thing I want to make clear. The effort to solve a Millennium Prize Problem, this was not due to multi-agent. I wouldn’t even attribute 10% of the credit to multi-agent. The reality is that OpenAI has trained a very powerful model. We can get that model to operate over very long horizons. We can get it to think in parallel.
But at its core, the reason why we’re able to do this is because we just have a general-purpose, very strong model. Things like multi-agent are flashy and new, and that probably gets disproportionate credit for that reason. But the core reason is this is just a very powerful model.
We trained GPT-5.6 end-to-end with three complementary architectural interventions that enable agents to operate more efficiently:
Reuse work already performed: by allowing reasoning to be persisted across model turns and using native compaction to compress long-running conversations, the model can maintain coherence in its work across longer task horizons without getting confused or having to reconstruct prior context.
Parallel decomposition where appropriate: using native multi-agent orchestration allows coordinating multiple agents across parallel workstreams to finish complex tasks faster.
Move deterministic work into code: using programmatic tool calling to filter, aggregate, and orchestrate tool outputs outside the model’s context window, reserving model tokens for judgment and reducing cost, latency, and context rot.
The infrastructure required to coordinate model weight updates with inference workers generating RL rollouts is complex. Nathan Lambert has written about some of the systems work required to make RL training efficient.
An oversimplified picture of an RL training system looks something like:
a set of learner workers updating the model weights;
a set of inference workers running some version of those weights to generate rollouts;
infrastructure moving weights, rewards, and other state between the two.
For a single-agent task, one rollout is a model producing a linear trajectory (tool calls, output tokens).
But once the policy being trained can spawn and coordinate other agents, the unit of a rollout becomes larger. Instead, one rollout becomes an asynchronous tree containing dozens or hundreds of model trajectories.
This means that the amount of inference compute that can usefully go into producing a single training trajectory can increase dramatically. And if you want the model to learn good coordination behavior - not merely bolt an orchestration layer onto an already-trained model - you want to include some of these large multi-agent trajectories inside the post-training loop.
I would guess the internal version of Astra is being trained with a refined recipe to encourage native multi-agent orchestration behavior. Even with this refined recipe, there are likely still quirks to work out, which is why Noam attributes <10% of the Navier-Stokes results to the multi-agent nature of the system they used to produce the solution.
As the recipe is improved, we'll see multi-agent systems behave more efficiently and produce more coherent behavior. Rather than spending spending 300B tokens to produce a solution, a more efficient multi-agent swarm may produce a solution with significantly fewer tokens.
As Noam also notes in the interview, this training process is still brittle. It is difficult to encourage the models to use cross-instance communication effectively; often you will see "useless" trajectories where models do not communicate and inject context into other instances at all.
Given the optimism and rumors of the new GPT-6 level model, though, it seems like the team at OpenAI has already made substantial progress in this direction.
Training models to coordinate across multiple agents gives us a way to productively spend substantially more compute during post-training than current single-agent RL setups.
From the recent Dwarkesh Podcast with Noam Brown:
Noam is referring to the GPT-5.6 announcement.
The OpenAI Builder's Guide to GPT-5.6 also hints at where multi-agent behavior enters the training stack (during RL post-training):
The infrastructure required to coordinate model weight updates with inference workers generating RL rollouts is complex. Nathan Lambert has written about some of the systems work required to make RL training efficient.
An oversimplified picture of an RL training system looks something like:
For a single-agent task, one rollout is a model producing a linear trajectory (tool calls, output tokens).
But once the policy being trained can spawn and coordinate other agents, the unit of a rollout becomes larger. Instead, one rollout becomes an asynchronous tree containing dozens or hundreds of model trajectories.
This means that the amount of inference compute that can usefully go into producing a single training trajectory can increase dramatically. And if you want the model to learn good coordination behavior - not merely bolt an orchestration layer onto an already-trained model - you want to include some of these large multi-agent trajectories inside the post-training loop.
I would guess the internal version of Astra is being trained with a refined recipe to encourage native multi-agent orchestration behavior. Even with this refined recipe, there are likely still quirks to work out, which is why Noam attributes <10% of the Navier-Stokes results to the multi-agent nature of the system they used to produce the solution.
As the recipe is improved, we'll see multi-agent systems behave more efficiently and produce more coherent behavior. Rather than spending spending 300B tokens to produce a solution, a more efficient multi-agent swarm may produce a solution with significantly fewer tokens.
As Noam also notes in the interview, this training process is still brittle. It is difficult to encourage the models to use cross-instance communication effectively; often you will see "useless" trajectories where models do not communicate and inject context into other instances at all.
Given the optimism and rumors of the new GPT-6 level model, though, it seems like the team at OpenAI has already made substantial progress in this direction.