In the current safety landscape, researchers are encouraged to use as many agents as possible, even in “fleets,” where a large number of them go out and run experiments while you, uh... wait for them to come back? I've spent the last few months of the Anthropic Fellows Program trying this out. My first attempts were naive, but after a few iterations I found an approach that works well for me.
Research is a double loop
I want to propose a mental model of research as a double loop, an idea that builds on double-loop learning (Argyris, 1977).
The outer loop is reserved for the inputs we currently refer to as "human taste": choosing which questions are worth answering, designing a trustworthy measure, and deciding which direction to go. Agents can help with this, but they’re still bad at making these decisions autonomously.
The inner loop turns each idea you test (an arm as claudius likes to say) into a comparable result. An example of a very simple inner loop might be:
Write a system prompt.
Run the model with that prompt on a set of test questions.
Score the model's replies.
Compare the scores against a control with no system prompt.
Plot results.
Agents are really good at building the infrastructure for this loop!
Before you can fully hand the inner loop to agents, you need to operationalize it. You need to work very closely to define the inputs so that you can freeze everything except the one input you’re testing (e.g. a system prompt). The benefit of this is that each arm fits in one small file that you can look at and compare easily, and running a new arm becomes running one command. I call research “fleetable” to the degree you can do this and check the results quickly.
What happens when you don’t operationalize
The naive way to use fleets is to hand agents the experiment and wait.
Let's look at an example. I asked five Claude Code agents to test whether a system prompt could reduce Claude Haiku 5.5's false refusals without making it more willing to answer harmful requests.
All five found that the prompt barely changed refusals, but three problems came up.
First, the results are hard to compare. Each agent made its own report and figure (Figure 2). You don't need to read the figures. They're all different, pretty ugly, and have some of the longest captions I've ever seen.
The reports don't even focus on the same thing. One agent found slightly more harmful content in the answers, while another found that the model answered harmful requests less often. Both can be true, but one makes the prompt look less safe and the other more safe.
First, the results are hard to compare. Each agent made its own report and figure which I've collaged below. You don't have to read the details of the figure - the point is that they all look different, and are frankly ugly, hard to read, and have the longest captions I've ever seen.
But it's not just an aesthetic issue; the reports don't even focus on the same thing! One agent found slightly more harmful content in the answers, while another found that the model answered harmful requests less often. Both can be true, but one makes the prompt look less safe and the other more safe.
To make matters worse, their numbers differ! Each agent chose its own judge, judge prompt, and number of samples per request. This led to a reported refusal rate anywhere from 21.0% to 27.8%!
Now imagine each agent testing a different prompt. How would you know if one was actually better, or just measured differently?
Second, agents make choices you wouldn’t. Agent 1 built a regex detector that looked for refusal phrases to check its judge. More concerningly, agent 5 tested the prompt with one sample per request, got p = 0.22, then added a second sample, increasing the risk of a false positive.
You only catch choices like these by looking the agents’ work and finding the dumb things humans would never do.
Third, you lose your understanding of the process. I’ve had people show me a figure their agents made, and when I asked about the details, they didn’t know where the numbers came from. I’m guilty of this too at times. Intimate involvement in the outer loop and heavy collaboration with agents on the inner loop can help fix this.
How to set up a fleet
So, what's the right way to do this? Here is the process I have converged on:
1. Freeze everything but the arm
Collaborate to write the pipeline once, in code, so every agent uses the same items, the same judge and judge prompt, the same number of samples, and the same figure. I use viz, my plotting library, so every figure comes out in one style. A good rule of thumb is “anything you need to correct an agent about belongs in the pipeline.”
An arm is then one small file: a system prompt, a short brief for generating training data, or a pair of prompts for a steering vector.
2. Brief each agent
Give each agent a goal and a direction to try. Tell it which file to write and which command to run. Here's an example for the experiment we ran:
Goal: a system prompt that makes Claude Haiku 5.5 refuse fewer requests that sound harmful but aren't, without complying more with requests that really are harmful. Your direction: assume the most plausible harmless reason someone would ask, and answer for that reason. 1. Write interventions/benign_intent.yaml (kind: prompt, a label, your system_prompt, control: control). Edit nothing else. 2. Run: uv run eval.py --out results/fleet_study --models anthropic:claude-haiku-5-5 \ --interventions benign_intent --repeat 2 --item-sets hard_search toxic_search 3. Reply with the path to figure.png and one sentence on what your arm shows.
3. Protect the pipeline
I keep the pipeline in files that agents run but never edit, and give the arms a folder of their own:
eval.py score.py train.py steer.py generate.py one script per step harness/ shared code, like the judge and the figure data/ the frozen test items interventions/ one file per arm, the only folder agents write to
Then I tell Claude Code to refuse edits to everything except the arms, in .claude/settings.json. The last rule stops agents from editing the rules themselves (which they absolutely love doing):
While building the pipeline, I found that my first judge was running out of tokens while thinking. It failed to label 35% to 55% of replies, but this wasn't obvious from the results. I only caught it by checking the raw outputs.
5. Add a manager agent
Once you have a fixed pipeline that works, you can run lots of agents in parallel to try different ideas. But to take this even further and really make it a fleet, you should setup a manager agent. It can turn your rough ideas into briefs and launch agents for you. You can even have it come up with ideas on its own, or use a separate ideas agent to send ideas to the manager. I suspect most of those experiments would have low EV for now, but may improve in the future.
What a fleet run looks like
Here's the same experiment with a fixed pipeline. I gave five agents different directions to try. Each wrote its own system prompt. One told the model to assume harmless intent. Another told it to add a caveat before helping. The naive agents later tested that second prompt.
The pipeline tested all five prompts against the same control. It used one Claude Opus 5.5 judge and counted both full and partial refusals. Here are the results that came back within about 10 minutes.
The more of the inner loop you can freeze, the more a fleet can do for you. The outer loop is still yours, at least for now.
In the current safety landscape, researchers are encouraged to use as many agents as possible, even in “fleets,” where a large number of them go out and run experiments while you, uh... wait for them to come back? I've spent the last few months of the Anthropic Fellows Program trying this out. My first attempts were naive, but after a few iterations I found an approach that works well for me.
Research is a double loop
I want to propose a mental model of research as a double loop, an idea that builds on double-loop learning (Argyris, 1977).
The outer loop is reserved for the inputs we currently refer to as "human taste": choosing which questions are worth answering, designing a trustworthy measure, and deciding which direction to go. Agents can help with this, but they’re still bad at making these decisions autonomously.
The inner loop turns each idea you test (an arm as claudius likes to say) into a comparable result. An example of a very simple inner loop might be:
Agents are really good at building the infrastructure for this loop!
Before you can fully hand the inner loop to agents, you need to operationalize it. You need to work very closely to define the inputs so that you can freeze everything except the one input you’re testing (e.g. a system prompt). The benefit of this is that each arm fits in one small file that you can look at and compare easily, and running a new arm becomes running one command. I call research “fleetable” to the degree you can do this and check the results quickly.
What happens when you don’t operationalize
The naive way to use fleets is to hand agents the experiment and wait.
Let's look at an example. I asked five Claude Code agents to test whether a system prompt could reduce Claude Haiku 5.5's false refusals without making it more willing to answer harmful requests.
All five found that the prompt barely changed refusals, but three problems came up.
First, the results are hard to compare. Each agent made its own report and figure (Figure 2). You don't need to read the figures. They're all different, pretty ugly, and have some of the longest captions I've ever seen.
The reports don't even focus on the same thing. One agent found slightly more harmful content in the answers, while another found that the model answered harmful requests less often. Both can be true, but one makes the prompt look less safe and the other more safe.
First, the results are hard to compare. Each agent made its own report and figure which I've collaged below. You don't have to read the details of the figure - the point is that they all look different, and are frankly ugly, hard to read, and have the longest captions I've ever seen.
But it's not just an aesthetic issue; the reports don't even focus on the same thing! One agent found slightly more harmful content in the answers, while another found that the model answered harmful requests less often. Both can be true, but one makes the prompt look less safe and the other more safe.
To make matters worse, their numbers differ! Each agent chose its own judge, judge prompt, and number of samples per request. This led to a reported refusal rate anywhere from 21.0% to 27.8%!
Now imagine each agent testing a different prompt. How would you know if one was actually better, or just measured differently?
Second, agents make choices you wouldn’t. Agent 1 built a regex detector that looked for refusal phrases to check its judge. More concerningly, agent 5 tested the prompt with one sample per request, got p = 0.22, then added a second sample, increasing the risk of a false positive.
You only catch choices like these by looking the agents’ work and finding the dumb things humans would never do.
Third, you lose your understanding of the process. I’ve had people show me a figure their agents made, and when I asked about the details, they didn’t know where the numbers came from. I’m guilty of this too at times. Intimate involvement in the outer loop and heavy collaboration with agents on the inner loop can help fix this.
How to set up a fleet
So, what's the right way to do this? Here is the process I have converged on:
1. Freeze everything but the arm
Collaborate to write the pipeline once, in code, so every agent uses the same items, the same judge and judge prompt, the same number of samples, and the same figure. I use viz, my plotting library, so every figure comes out in one style. A good rule of thumb is “anything you need to correct an agent about belongs in the pipeline.”
An arm is then one small file: a system prompt, a short brief for generating training data, or a pair of prompts for a steering vector.
2. Brief each agent
Give each agent a goal and a direction to try. Tell it which file to write and which command to run. Here's an example for the experiment we ran:
Goal: a system prompt that makes Claude Haiku 5.5 refuse fewer requests that sound harmfulbut aren't, without complying more with requests that really are harmful.
Your direction: assume the most plausible harmless reason someone would ask, and answer for that reason.
1. Write interventions/benign_intent.yaml (kind: prompt, a label, your system_prompt, control: control).
Edit nothing else.
2. Run: uv run eval.py --out results/fleet_study --models anthropic:claude-haiku-5-5 \
--interventions benign_intent --repeat 2 --item-sets hard_search toxic_search
3. Reply with the path to figure.png and one sentence on what your arm shows.
3. Protect the pipeline
I keep the pipeline in files that agents run but never edit, and give the arms a folder of their own:
eval.py score.py train.py steer.py generate.py one script per stepharness/ shared code, like the judge and the figure
data/ the frozen test items
interventions/ one file per arm, the only folder agents write to
Then I tell Claude Code to refuse edits to everything except the arms, in
.claude/settings.json. The last rule stops agents from editing the rules themselves (which they absolutely love doing):{"permissions": {
"deny": [
"Edit(/eval.py)",
"Edit(/score.py)",
"Edit(/train.py)",
"Edit(/steer.py)",
"Edit(/generate.py)",
"Edit(/harness/**)",
"Edit(/data/**)",
"Edit(/.claude/**)"
]
}
}
4. Read the raw outputs yourself
While building the pipeline, I found that my first judge was running out of tokens while thinking. It failed to label 35% to 55% of replies, but this wasn't obvious from the results. I only caught it by checking the raw outputs.
5. Add a manager agent
Once you have a fixed pipeline that works, you can run lots of agents in parallel to try different ideas. But to take this even further and really make it a fleet, you should setup a manager agent. It can turn your rough ideas into briefs and launch agents for you. You can even have it come up with ideas on its own, or use a separate ideas agent to send ideas to the manager. I suspect most of those experiments would have low EV for now, but may improve in the future.
What a fleet run looks like
Here's the same experiment with a fixed pipeline. I gave five agents different directions to try. Each wrote its own system prompt. One told the model to assume harmless intent. Another told it to add a caveat before helping. The naive agents later tested that second prompt.
The pipeline tested all five prompts against the same control. It used one Claude Opus 5.5 judge and counted both full and partial refusals. Here are the results that came back within about 10 minutes.
The more of the inner loop you can freeze, the more a fleet can do for you. The outer loop is still yours, at least for now.