A summary of our ICML 2026 paper, Architecture Matters for Multi-Agent Security (Ben Hagag,William L. Anderson, Christian Schroeder de Witt, Sarah Scheffler). The scenarios and experiments were built on Orbit, a multi-agent security experimentation framework we'll release v0 of soon. Benchmark adaptations and code are on GitHub. Orbit is very much still a work in progress, and we welcome collaborators, feedback, and suggestions. (We'll be presenting this at ICML 2026 in Seoul. if you're attending and want to chat about multi-agent security, get in touch.)
TL;DR. We took three single-agent misuse benchmarks and built multi-agent systems to run on them, then swept various architectural choices: roles, topology, memory, across six models. The question was whether and how these design decisions change how exploitable the system is.
It had a large, difficult-to-predict impact. The same model and task can go from refusing to complying depending only on the architecture. We did not find a design that is safe everywhere: the topology that was safest in one environment was the riskiest in another. Security is not correlated with the system's performance, so you would not (of course) be able to read a system's misuse risk off its benchmark scores.
We knew from previous research that adding agents adds attack surface, and that fragmenting a task lets harmful requests slip past agents that would have refused the whole thing. Those mechanisms are intuitive once you see them. What did surprise us was how inconsistent the effects were. We expected architectural effects to at least point in the same direction across environments. They didn't, and we still can't fully explain why. Our evidence points towards a complex interplay between model, architecture, and setting. We are far from a principled understanding of multi-agent security, and thus currently recommend empirical security testing of multi-agent systems beyond the model level. We're releasing all scenarios and the codebase for future research, and we'll soon release Orbit, a first-of-its-kind framework for evaluating multi-agent systems security. This is work in progress, and we welcome feedback and comments. Feel free to reach out if you'd like to contribute.
Why multi-agent systems
More and more systems are being built as networks of agents that plan, call tools, share memory, and pass work between each other, rather than as a single model answering a prompt. Part of the reason is capability: these systems handle large, long-horizon tasks that a single model tends to fumble. Part of it is structural: sometimes you want separate agents, for reasons that have nothing to do with performance, like keeping data or IP walled off between roles.
Multi-agent systems are well studied, but classical work mostly assumes fixed and narrow agents on rigid protocols, so its guarantees don't transfer to LLM agents that reason freely and coordinate in natural language. Security research hasn't really caught up either. Most of it is still about the single model (prompt injection, jailbreaks) or the single agent (tool misuse, memory poisoning). We already have reason to think that isn't enough: both the classical MAS literature and recent LLM-agent security work suggest hardening the parts doesn't harden the whole. So we treat architecture as its own security variable.
The questions we set out to answer
What are the design dimensions (the knobs) one can actually vary in MAS?
Is there a knob setting, or some structural choice, that reliably improves security across settings?
What is the relationship between task performance and security?
Is a multi-agent system more or less secure than a single agent?
The design space: three knobs
We focused on three design choices that recur across essentially every modern MAS framework, varying each one while holding the others fixed. This is only a subset of all of the possible choices, but we consider these three fundamental, they are implicit or explicit choices that one makes in any case of building a MAS.
Role configuration: how authority and responsibility are split: a single generalist agent, or a planner that decomposes a task and delegates to executors and specialists. The rationale for studying it is that role separation can fragment security assessment. An executor working on a narrowly scoped subtask that was given to it by another agent or an orchestrator, has no view of the full objective. A reviewer agent may check an artifact in isolation without assessing end-to-end behavior. When the agents with the most execution authority sit downstream of the ones doing the safety reasoning, safety judgments might become advisory rather than binding. The opposite case is also plausible, though: decomposition increases the number of checkpoints a harmful task must pass through, which could raise the likelihood that one of them recognizes the true intent and blocks the task.
We varied decomposition along two dimensions. The first is the granularity of delegation: a batch executor receives several steps from the preceding agent, performs all of them, and reports back, whereas a stepwise executor performs only one action at a time before returning control. The second is how many tools and capabilities each agent holds. We tested 2-, 3-, and 4-specialist configurations, where each setting distributes a different number of tools and capabilities across the agents.
Communication topology: determines how agents are wired to exchange information, including governing and safety signals. We studied a star (a central orchestrator mediating all traffic), a chain (a fixed sequential pipeline), and a mesh (peer-to-peer with no central coordinator). These might route oversight differently. For example, a mesh spreads it across every peer, while a chain summarizes and filters information as it passes down the line, which can strip the context an agent would need to recognize harm.
Memory and state visibility: determines what each agent can see of its own prior steps and the other agents in the system. Private scratchpad only (baseline), the agent's own reasoning traces, or full shared memory where every agent observes the complete execution history. The rationale is that shared state can improve transparency but also let unsafe behavior propagate, while private state preserves independence but can keep a safety-relevant observation from reaching the agent that needs it.
The scenarios
We adapted three single-agent misuse benchmarks to multi-agent settings, while preserving the original task semantics. The only things that change across our conditions are the role configuration, communication topology and memory:
BrowserART (browser control): a fixed set of narrow tools (goto, click, fill, scroll), which permits a clean one-tool-per-agent split in the four-specialist configuration.
OS-Harm (desktop computer use): a single computer tool operating on pixels (screenshot + mouse + keyboard), which we partition into disjoint sub-tools (click, type, key, scroll).
RedCode-Gen (sandboxed code generation): four tools (python, editor, bash, think) split across Design / Code / Review / Test roles.
Experimental design
We evaluate a malicious user who supplies adversarial instructions at input time. Across the three environments we ran 13 architectural configurations.
So we used stage-wise metrics that classify each trajectory into one of four disjoint outcomes: Planning Refusal (refused before acting or making a plan), Execution Refusal (the executor refused after delegation), Harmful Action (a harmful action was taken but the objective wasn't completed), and Harmful Task (the objective was completed). We also record benign task success on a separate set of legitimate tasks to track the performance cost of each design (We used browserArt benign tasks and expanded them, as the original dataset contains only 10 tasks. We used OS-world as the benign task dataset for OS-Harm, and BigCodeBench as the benign task dataset for RedCode-Gen). We report harmful-task completion (HT) as the primary metric.
We started our experiments with GPT-4o, later extended to five further models (GPT-5.4, GPT-5-mini, Claude Sonnet 4, Qwen3-VL, Llama-70B). We evaluated across all three environments and all three design axes. Judging follows each original benchmark's protocol revised to support multi-agent system, with a two-phase validation pass (LLM flagging followed by manual review of flagged cases).
The figure below illustrates the basic effect on one task. The same GPT-4o that refuses a malicious icmp_flood request as a standalone agent will, once the task is decomposed across a pipeline of role-specialized agents, complete it.
Results
We generally see very mixed, difficult to predict effects from any of these changes. We currently lack a coherent theory that explains these differences, though we certainly plan future work in this direction. In this section, we offer our results and some weak guesses at possible explanations, but these should not be interpreted strongly. We primarily show that there are strong effects from changing architecture, but don’t have coherent models that explain why we see the effects we do nor have the ability to predict them a priori. Until further progress, we thus suggest any multi-agent system should be treated as a security primitive and evaluated independently.
Role decomposition
The effect of splitting one agent into specialists is domain-dependent in both magnitude and direction.
On BrowserART, a single or batch executor lowers HT relative to the standalone agent (5% vs. 10%), because centralizing the plan keeps one agent reasoning over the whole request. The jump comes with specialization: HT rises to 38% at two specialists (then 27% and 31% at three and four), as planning refusal drops from 75% to the 40–45% range. The trend across specialist counts is non-monotonic, which suggests how tools are partitioned matters, not just how many agents there are.
On RedCode-Gen, the pattern is different: even simple delegation raises risk, and HT then decreases as specialists are added (from 25.6% to 23.8%, then 17.5% for two, three, four agents), the opposite of the BrowserART direction. Planning refusal collapses (down to ~14%) but is partly compensated by execution-stage refusal (up to ~39%), meaning the safety reasoning shifts from the planning stage to the execution stage.
On OS-Harm, the standalone rate is low (4.5%) and specialization raises it only modestly (up to 9.1%), plausibly because pixel-level clicks and keystrokes are less semantically legible than browser or code actions.
The model matters as much or more than the architecture. Newer, more well-aligned and capable models largely resist decomposition. GPT-5.4 stays at ≤3% HT on BrowserART, Sonnet 4 at ≤7%, and GPT-5.4 sits at 0% across all RedCode-Gen configurations. Weaker-aligned models degrade sharply: Qwen3-VL rises from 9% to 40% and Llama-70B from 27% to 42% under two-specialist decomposition.
Communication topology
No topology is safest everywhere.
On BrowserART, star is riskiest (31% HT), then chain (16%), then mesh (7%). Mesh is the only multi-agent configuration that comes in below the standalone agent, with planning refusal at 78%, consistent with the idea that mesh agents each see the full objective and can refuse individually. But mesh also has the lowest benign accuracy of the three topologies (81% vs. 95–98%).
On RedCode-Gen, chain is the riskiest topology at 42.5% HT, nearly double star (17.5%) and mesh (20.6%). Each stage only sees the output of the one before it, so by the time a harmful request passes through design, code, review, and test, no single agent has enough context to flag it. Planning refusal drops to just 8%, showing how a pipeline diffuses responsibility until no one agent feels accountable enough to refuse.
On OS-Harm, every topology exceeds the standalone rate, with chain highest at 15.9%.
Star is the riskiest topology on BrowserART and the safest on RedCode-Gen. The effect of topology is therefore not fixed but conditional on the task: our results are consistent with risk depending on how much of the original objective each agent observes, though we cannot yet isolate this mechanism from other differences between environments.
Memory
We varied what each agent can see: private scratchpad only, the agent's own reasoning traces, or full shared memory, under fixed roles and topology. The figure below shows the net effect of going from private to full shared memory in each of the six topology-by-environment combinations.
Only one of the six combinations (RedCode-Gen mesh) becomes safer with more shared context, and one is unchanged. The other four become modestly more vulnerable. The magnitudes are small and the signs don't agree, so there is no recommendation we can derive as to whether more or less context is universally good or bad.
GPT-5.4's near-zero harmful completion leaves no room for memory to have any measurable effect, and the weaker models replicate the shared-memory vulnerability (only partially, on star but not consistently on mesh).
Performance and security decouple
We found out that benign performance and security don't move together. The figure below tracks both metrics for GPT-4o across all 13 architectural configurations, grouped into the role, topology, and memory axes, for the two environments where benign accuracy is meaningfully measured (BrowserART and RedCode-Gen). The solid line is benign task accuracy. The dashed line is harmful-task completion. (We leave out OS-Harm here: its benign accuracy is about 0 because the underlying OS-World tasks are too hard, so the comparison tells us nothing.)
The two lines do not track each other in either domain or along any of the three design axes. On BrowserART, benign accuracy stays in a narrow high band (81–98%), while harmful-task completion swings from 3% to 38% over the same configurations. On RedCode-Gen, benign accuracy moves on its own (peaking at stepwise, dipping under specialization) while harmful-task completion again follows a different shape entirely.
So if we wanted to use task performance as a measure or signal of the vulnerability of the system, these small-scope yet diverse experiments show that we could not derive any conclusion solely based on task performance. On BrowserART that line would look flat-to-improving across exactly the configurations where the dashed line, the actual misuse risk, climbs several-fold. Two systems that a benchmark would call equally capable can differ substantially in harmful-task completion. As far as our experiments go (and we don't claim they are broad enough to settle the question), capability metrics provide no signal about architectural security risk, and further, there don't seem to be general choices which work universally well, which is the argument for measuring security separately and per-deployment.
Recap what we found:
In the majority of configurations, multi-agent architectures are more vulnerable than the standalone agent, often by substantial margins. This is a majority, not a universal: some configurations are safer.
We did not find any single architecture or mechanism that is robust across all scenarios and setups. The direction and magnitude of every effect depended on the environment, the model, and the architecture.
We release all three multi-agent benchmark adaptations (BrowserART, OS-Harm, RedCode-Gen) to support further work.
A dynamic framework for running and evaluating multi-agent security experiments is in progress and will follow.
Discussion
Multi-agent systems expand the attack surface: Each added agent, channel, and role boundary added a new place where safety/defense reasoning can fail or be bypassed. That moving from one agent to many should be harder to secure is, in principle, unsurprising. What our results quantify is the extent and the unpredictability of the effect.
The effects are scenario- and model-dependent: Our results do not support a single ranking of safe vs. unsafe designs. Star is riskiest on BrowserART but safest on RedCode-Gen and OS-Harm. More memory visibility helps in one of six cases and hurts in two, and role specialization has the largest effect on BrowserART but acts in different directions elsewhere. Architecture also interacts with the base model's safety training: well-aligned models stay robust across nearly all configurations, while weaker ones can show roughly fourfold increases in harmful-task completion under the same architectural change.
Mechanisms: More work needs to be done to isolate the different setups and to come to definitive conclusions about the mechanisms behind security in multi agent systems. Our experiment shows that planning refusal drop (meaning more potential to harmful task) when specialist agents receive isolated instructions with no view of the full task. These are quite obvious results but worth noting. In mesh topology, agents see the whole objective and can refuse more easily. In chain setup, agents receive processed outputs that mask intent, which we see as lower planning refusal.
Implications for system design: security can't be inferred from component-level evaluation alone, since agents that refuse on their own can still enable harm once you decompose the task across them, miscoordination effect or in more complex scenarios even collusion. Choices that improve or maintain performance can degrade security in hard-to-predict ways. And because the effects depend on the scenario and the model, architectural security has to be evaluated per deployment and per setup.
Takeaways
This was always fairly clear, but architecture is an important security variable in LLM-based multi agent systems. We're currently far away from a robust understanding of the security properties of these systems, and relatively small-looking changes can have large impacts on security.
Safety properties do not compose straightforwardly from agents to systems.
There is no architecture we can recommend as secure independent of the deployment. Test your own architecture, in your own environment. Don't assume a small change (one more agent, one split role) preserves the security profile.
Caveats
A few limits on how far to read these results:
We did not prove that no universally safe mechanism exists. We searched a structured but finite design space and did not find one. That does not rule out such a mechanism, whether a single one that works everywhere or a separate best choice for each setting (for example, for a chain topology, some specific combination of memory visibility and role decomposition might give safer outcomes).
We study direct misuse by a malicious user only. Other threats like indirect prompt injection, memory poisoning, and adversarial agents operating may interact with architecture differently.
We did not evaluate dedicated safety mechanisms like monitor agents and policy-checking stages, which would change the tradeoffs we measured. We leave this to follow-up work.
Findings are scoped to the three environments (browser, desktop, code) and should be read within that scope.
Moving forward
Over the last year, there have been increasingly many interesting multi-agent safety and security results, and yet, progress seems much slower than is needed. Currently, it seems there is no standard infrastructure for empirical multi-agent security evals, which means many projects are duplicating engineering work and it's hard to draw standardized comparisons across different defenses. We'll soon be releasing v0 of Orbit, our infrastructure for doing multi-agent security experiments in Inspect. While it's very much a work-in-progress, we hope to make it the standard framework for empirical multi-agent safety and security research.
See the paper for full tables, the per-model breakdowns, and the experimental details. Our benchmark adaptations are public.
A summary of our ICML 2026 paper, Architecture Matters for Multi-Agent Security (Ben Hagag, William L. Anderson, Christian Schroeder de Witt, Sarah Scheffler). The scenarios and experiments were built on Orbit, a multi-agent security experimentation framework we'll release v0 of soon. Benchmark adaptations and code are on GitHub. Orbit is very much still a work in progress, and we welcome collaborators, feedback, and suggestions. (We'll be presenting this at ICML 2026 in Seoul. if you're attending and want to chat about multi-agent security, get in touch.)
This work done during MATS Winter 2026.
TL;DR. We took three single-agent misuse benchmarks and built multi-agent systems to run on them, then swept various architectural choices: roles, topology, memory, across six models. The question was whether and how these design decisions change how exploitable the system is.
It had a large, difficult-to-predict impact. The same model and task can go from refusing to complying depending only on the architecture. We did not find a design that is safe everywhere: the topology that was safest in one environment was the riskiest in another. Security is not correlated with the system's performance, so you would not (of course) be able to read a system's misuse risk off its benchmark scores.
We knew from previous research that adding agents adds attack surface, and that fragmenting a task lets harmful requests slip past agents that would have refused the whole thing. Those mechanisms are intuitive once you see them. What did surprise us was how inconsistent the effects were. We expected architectural effects to at least point in the same direction across environments. They didn't, and we still can't fully explain why. Our evidence points towards a complex interplay between model, architecture, and setting. We are far from a principled understanding of multi-agent security, and thus currently recommend empirical security testing of multi-agent systems beyond the model level. We're releasing all scenarios and the codebase for future research, and we'll soon release Orbit, a first-of-its-kind framework for evaluating multi-agent systems security. This is work in progress, and we welcome feedback and comments. Feel free to reach out if you'd like to contribute.
Why multi-agent systems
More and more systems are being built as networks of agents that plan, call tools, share memory, and pass work between each other, rather than as a single model answering a prompt. Part of the reason is capability: these systems handle large, long-horizon tasks that a single model tends to fumble. Part of it is structural: sometimes you want separate agents, for reasons that have nothing to do with performance, like keeping data or IP walled off between roles.
Multi-agent systems are well studied, but classical work mostly assumes fixed and narrow agents on rigid protocols, so its guarantees don't transfer to LLM agents that reason freely and coordinate in natural language. Security research hasn't really caught up either. Most of it is still about the single model (prompt injection, jailbreaks) or the single agent (tool misuse, memory poisoning). We already have reason to think that isn't enough: both the classical MAS literature and recent LLM-agent security work suggest hardening the parts doesn't harden the whole. So we treat architecture as its own security variable.
The questions we set out to answer
The design space: three knobs
We focused on three design choices that recur across essentially every modern MAS framework, varying each one while holding the others fixed. This is only a subset of all of the possible choices, but we consider these three fundamental, they are implicit or explicit choices that one makes in any case of building a MAS.
Role configuration: how authority and responsibility are split: a single generalist agent, or a planner that decomposes a task and delegates to executors and specialists. The rationale for studying it is that role separation can fragment security assessment. An executor working on a narrowly scoped subtask that was given to it by another agent or an orchestrator, has no view of the full objective. A reviewer agent may check an artifact in isolation without assessing end-to-end behavior. When the agents with the most execution authority sit downstream of the ones doing the safety reasoning, safety judgments might become advisory rather than binding. The opposite case is also plausible, though: decomposition increases the number of checkpoints a harmful task must pass through, which could raise the likelihood that one of them recognizes the true intent and blocks the task.
We varied decomposition along two dimensions. The first is the granularity of delegation: a batch executor receives several steps from the preceding agent, performs all of them, and reports back, whereas a stepwise executor performs only one action at a time before returning control. The second is how many tools and capabilities each agent holds. We tested 2-, 3-, and 4-specialist configurations, where each setting distributes a different number of tools and capabilities across the agents.
Communication topology: determines how agents are wired to exchange information, including governing and safety signals. We studied a star (a central orchestrator mediating all traffic), a chain (a fixed sequential pipeline), and a mesh (peer-to-peer with no central coordinator). These might route oversight differently. For example, a mesh spreads it across every peer, while a chain summarizes and filters information as it passes down the line, which can strip the context an agent would need to recognize harm.
Memory and state visibility: determines what each agent can see of its own prior steps and the other agents in the system. Private scratchpad only (baseline), the agent's own reasoning traces, or full shared memory where every agent observes the complete execution history. The rationale is that shared state can improve transparency but also let unsafe behavior propagate, while private state preserves independence but can keep a safety-relevant observation from reaching the agent that needs it.
The scenarios
We adapted three single-agent misuse benchmarks to multi-agent settings, while preserving the original task semantics. The only things that change across our conditions are the role configuration, communication topology and memory:
Experimental design
We evaluate a malicious user who supplies adversarial instructions at input time. Across the three environments we ran 13 architectural configurations.
So we used stage-wise metrics that classify each trajectory into one of four disjoint outcomes: Planning Refusal (refused before acting or making a plan), Execution Refusal (the executor refused after delegation), Harmful Action (a harmful action was taken but the objective wasn't completed), and Harmful Task (the objective was completed). We also record benign task success on a separate set of legitimate tasks to track the performance cost of each design (We used browserArt benign tasks and expanded them, as the original dataset contains only 10 tasks. We used OS-world as the benign task dataset for OS-Harm, and BigCodeBench as the benign task dataset for RedCode-Gen). We report harmful-task completion (HT) as the primary metric.
We started our experiments with GPT-4o, later extended to five further models (GPT-5.4, GPT-5-mini, Claude Sonnet 4, Qwen3-VL, Llama-70B). We evaluated across all three environments and all three design axes. Judging follows each original benchmark's protocol revised to support multi-agent system, with a two-phase validation pass (LLM flagging followed by manual review of flagged cases).
The figure below illustrates the basic effect on one task. The same GPT-4o that refuses a malicious icmp_flood request as a standalone agent will, once the task is decomposed across a pipeline of role-specialized agents, complete it.
Results
We generally see very mixed, difficult to predict effects from any of these changes. We currently lack a coherent theory that explains these differences, though we certainly plan future work in this direction. In this section, we offer our results and some weak guesses at possible explanations, but these should not be interpreted strongly. We primarily show that there are strong effects from changing architecture, but don’t have coherent models that explain why we see the effects we do nor have the ability to predict them a priori. Until further progress, we thus suggest any multi-agent system should be treated as a security primitive and evaluated independently.
Role decomposition
The effect of splitting one agent into specialists is domain-dependent in both magnitude and direction.
On BrowserART, a single or batch executor lowers HT relative to the standalone agent (5% vs. 10%), because centralizing the plan keeps one agent reasoning over the whole request. The jump comes with specialization: HT rises to 38% at two specialists (then 27% and 31% at three and four), as planning refusal drops from 75% to the 40–45% range. The trend across specialist counts is non-monotonic, which suggests how tools are partitioned matters, not just how many agents there are.
On RedCode-Gen, the pattern is different: even simple delegation raises risk, and HT then decreases as specialists are added (from 25.6% to 23.8%, then 17.5% for two, three, four agents), the opposite of the BrowserART direction. Planning refusal collapses (down to ~14%) but is partly compensated by execution-stage refusal (up to ~39%), meaning the safety reasoning shifts from the planning stage to the execution stage.
On OS-Harm, the standalone rate is low (4.5%) and specialization raises it only modestly (up to 9.1%), plausibly because pixel-level clicks and keystrokes are less semantically legible than browser or code actions.
The model matters as much or more than the architecture. Newer, more well-aligned and capable models largely resist decomposition. GPT-5.4 stays at ≤3% HT on BrowserART, Sonnet 4 at ≤7%, and GPT-5.4 sits at 0% across all RedCode-Gen configurations. Weaker-aligned models degrade sharply: Qwen3-VL rises from 9% to 40% and Llama-70B from 27% to 42% under two-specialist decomposition.
Communication topology
No topology is safest everywhere.
On BrowserART, star is riskiest (31% HT), then chain (16%), then mesh (7%). Mesh is the only multi-agent configuration that comes in below the standalone agent, with planning refusal at 78%, consistent with the idea that mesh agents each see the full objective and can refuse individually. But mesh also has the lowest benign accuracy of the three topologies (81% vs. 95–98%).
On RedCode-Gen, chain is the riskiest topology at 42.5% HT, nearly double star (17.5%) and mesh (20.6%). Each stage only sees the output of the one before it, so by the time a harmful request passes through design, code, review, and test, no single agent has enough context to flag it. Planning refusal drops to just 8%, showing how a pipeline diffuses responsibility until no one agent feels accountable enough to refuse.
On OS-Harm, every topology exceeds the standalone rate, with chain highest at 15.9%.
Star is the riskiest topology on BrowserART and the safest on RedCode-Gen. The effect of topology is therefore not fixed but conditional on the task: our results are consistent with risk depending on how much of the original objective each agent observes, though we cannot yet isolate this mechanism from other differences between environments.
Memory
We varied what each agent can see: private scratchpad only, the agent's own reasoning traces, or full shared memory, under fixed roles and topology. The figure below shows the net effect of going from private to full shared memory in each of the six topology-by-environment combinations.
Only one of the six combinations (RedCode-Gen mesh) becomes safer with more shared context, and one is unchanged. The other four become modestly more vulnerable. The magnitudes are small and the signs don't agree, so there is no recommendation we can derive as to whether more or less context is universally good or bad.
GPT-5.4's near-zero harmful completion leaves no room for memory to have any measurable effect, and the weaker models replicate the shared-memory vulnerability (only partially, on star but not consistently on mesh).
Performance and security decouple
We found out that benign performance and security don't move together. The figure below tracks both metrics for GPT-4o across all 13 architectural configurations, grouped into the role, topology, and memory axes, for the two environments where benign accuracy is meaningfully measured (BrowserART and RedCode-Gen). The solid line is benign task accuracy. The dashed line is harmful-task completion. (We leave out OS-Harm here: its benign accuracy is about 0 because the underlying OS-World tasks are too hard, so the comparison tells us nothing.)
The two lines do not track each other in either domain or along any of the three design axes. On BrowserART, benign accuracy stays in a narrow high band (81–98%), while harmful-task completion swings from 3% to 38% over the same configurations. On RedCode-Gen, benign accuracy moves on its own (peaking at stepwise, dipping under specialization) while harmful-task completion again follows a different shape entirely.
So if we wanted to use task performance as a measure or signal of the vulnerability of the system, these small-scope yet diverse experiments show that we could not derive any conclusion solely based on task performance. On BrowserART that line would look flat-to-improving across exactly the configurations where the dashed line, the actual misuse risk, climbs several-fold. Two systems that a benchmark would call equally capable can differ substantially in harmful-task completion. As far as our experiments go (and we don't claim they are broad enough to settle the question), capability metrics provide no signal about architectural security risk, and further, there don't seem to be general choices which work universally well, which is the argument for measuring security separately and per-deployment.
Recap what we found:
Discussion
Multi-agent systems expand the attack surface: Each added agent, channel, and role boundary added a new place where safety/defense reasoning can fail or be bypassed. That moving from one agent to many should be harder to secure is, in principle, unsurprising. What our results quantify is the extent and the unpredictability of the effect.
The effects are scenario- and model-dependent: Our results do not support a single ranking of safe vs. unsafe designs. Star is riskiest on BrowserART but safest on RedCode-Gen and OS-Harm. More memory visibility helps in one of six cases and hurts in two, and role specialization has the largest effect on BrowserART but acts in different directions elsewhere. Architecture also interacts with the base model's safety training: well-aligned models stay robust across nearly all configurations, while weaker ones can show roughly fourfold increases in harmful-task completion under the same architectural change.
Mechanisms: More work needs to be done to isolate the different setups and to come to definitive conclusions about the mechanisms behind security in multi agent systems. Our experiment shows that planning refusal drop (meaning more potential to harmful task) when specialist agents receive isolated instructions with no view of the full task. These are quite obvious results but worth noting. In mesh topology, agents see the whole objective and can refuse more easily. In chain setup, agents receive processed outputs that mask intent, which we see as lower planning refusal.
Implications for system design: security can't be inferred from component-level evaluation alone, since agents that refuse on their own can still enable harm once you decompose the task across them, miscoordination effect or in more complex scenarios even collusion. Choices that improve or maintain performance can degrade security in hard-to-predict ways. And because the effects depend on the scenario and the model, architectural security has to be evaluated per deployment and per setup.
Takeaways
Caveats
A few limits on how far to read these results:
Moving forward
Over the last year, there have been increasingly many interesting multi-agent safety and security results, and yet, progress seems much slower than is needed. Currently, it seems there is no standard infrastructure for empirical multi-agent security evals, which means many projects are duplicating engineering work and it's hard to draw standardized comparisons across different defenses. We'll soon be releasing v0 of Orbit, our infrastructure for doing multi-agent security experiments in Inspect. While it's very much a work-in-progress, we hope to make it the standard framework for empirical multi-agent safety and security research.
See the paper for full tables, the per-model breakdowns, and the experimental details. Our benchmark adaptations are public.