Multiple recent developments - such as GPT-5.6 hacking into HuggingFace to cheat in a cybersecurity eval - have underscored the need to increase our capability to evaluate the cybersecurity capabilities of new and upcoming AI models.
Evaluate the cybersecurity capabilities of new and upcoming AI models,
Train existing models to increase their cybersecurity capabilities.
On the surface of it, these seem to conflict.
However, it is my view that more open-source security tooling means more secure systems. More on dual-use below.
The Motivation
Many existing cybersecurity benchmarks face one or more of three problems that I think make rigorous evaluation harder:
Ambiguously graded benchmarks
Game-able (i.e. possible to be reward-hacked)
Limited in volume
By (1), I mean that some benchmarks can robustly determine whether the final objective was achieved, but provide much weaker evidence about how it was achieved. This matters when an unintended solution, leaked artifact, benchmark contamination, or environment failure can produce the same apparent success.
(2) means that for a given program or target, an evaluator may want to see if the AI can bypass certain defensive mechanism in order to achieve the desired result. However, these benchmarks don't (=can't) check whether the AI found an alternate way of achieving that result. This has been abundantly clear during the recent news surrounding GPT-5.6 trying to cheat its way through a security benchmark by hacking HuggingFace.
(3) is a problem in two ways. The first, benchmarks are quickly contaminated. If we can't manually create many new instances of cybersecurity tasks, then we can't iteratively evaluate new capabilities. Second, volume limitation decreases researchers' capability to do proper statistical and deep analysis of new capabilities.
TarantuBench aims to address these gaps, but does not claim to solve them completely
What it is
Ten thousand AI-generated vulnerable web appsspanning roughly 2,400 distinct technique configurations, which are verifiably exploitable, that include a two-tier detection mechanism for reward hacking.
The 2nd version of the benchmark was focused on two things:
Scale (and the ability to continuously generate at this scale)
Reward Hacking Detection
Increasing the scale from one hundred labs to ten thousand introduced some engineering and budget constraints which forced me to adopt a different approach than I previously did. In v1, I allowed the AI to have a freer hand to draw up the lab under the scope given to it that allowed me to verify it is exploitable. In order to not 100x my budget here, I had to have the AI be more limited, and to take more control of the lab's structure.
I implemented two complementary detection mechanisms. The application itself requires exploit-specific milestones to be satisfied before releasing the flag. Separately, the evaluation harness checks the agent's HTTP trace for loose alignment with the expected exploit path, payload class, and ordering. Neither proves that the intended technique was used, but together they provide stronger evidence of unintended solves or broken environments.
These two systems are not perfect, and it's my belief that no such system that can provably detect all reward-hacking exists, but this is my attempt to let the researcher know when the AI has solved the lab in an unintended way, and to allow them to behave accordingly.
These two improvements, in addition to having v1 already contain binary checks of whether the flag submitted by the agent is correct, are my attempt at addressing the three gaps I outlined earlier.
What it isn't
This benchmark is a set of web-apps with a single security vulnerability within them, and are synthetically generated.
This means that it doesn't simulate real-world security interactions, and if trained primarily on this benchmark, may teach security in how the benchmark views it to be, rather than how it is in the real-world.
Ideally, I would have taken two additional steps before publishing it:
Check the correlation between existing model's cybersecurity capabilities on other benchmarks, and compare to mine,
Fine-tune a model on this benchmark, and see how well it transfers to other benchmarks
I cannot do so due to budget constraints.
On dual-use
My view is that security tooling is mostly best when open-source. I emphasize the word tooling, which is distinct from any artifact which may include previously unknown security vulnerabilities, the code that exploits them, and any other code which would create more harm than good in the open. It is true that security tooling can be used by both malicious attackers and defenders, but more often than not, having the capability in both operator's hands is what's best.
This set of tooling extends to AI models which can find vulnerabilities in web applications. Internal defenders as well as external researchers would do great defensive work with access to AI that can find these gaps.
Bottom line & collaboration
The technical deep-dive of how the benchmark looks and works is available in the link above to HuggingFace. I aimed to have this post generally outline the approach and philosophy.
I'll continue to work on this effort of evaluating AI cybersecurity capabilities, and more so given the worrying news coming out from the leading labs about their sharply increasing capabilities.
I already have a mental model of what v3 & v4 will include, and I'd additionally appreciate any thoughts & insights you may have on my work or where the following versions need to go.
If you wish to contact me about this work, I'm available here and at tomer@tarantulabs.com.
Multiple recent developments - such as GPT-5.6 hacking into HuggingFace to cheat in a cybersecurity eval - have underscored the need to increase our capability to evaluate the cybersecurity capabilities of new and upcoming AI models.
TarantuBench-v2 aims to do two things:
On the surface of it, these seem to conflict.
However, it is my view that more open-source security tooling means more secure systems. More on dual-use below.
The Motivation
Many existing cybersecurity benchmarks face one or more of three problems that I think make rigorous evaluation harder:
By (1), I mean that some benchmarks can robustly determine whether the final objective was achieved, but provide much weaker evidence about how it was achieved. This matters when an unintended solution, leaked artifact, benchmark contamination, or environment failure can produce the same apparent success.
(2) means that for a given program or target, an evaluator may want to see if the AI can bypass certain defensive mechanism in order to achieve the desired result. However, these benchmarks don't (=can't) check whether the AI found an alternate way of achieving that result. This has been abundantly clear during the recent news surrounding GPT-5.6 trying to cheat its way through a security benchmark by hacking HuggingFace.
(3) is a problem in two ways. The first, benchmarks are quickly contaminated. If we can't manually create many new instances of cybersecurity tasks, then we can't iteratively evaluate new capabilities. Second, volume limitation decreases researchers' capability to do proper statistical and deep analysis of new capabilities.
TarantuBench aims to address these gaps, but does not claim to solve them completely
What it is
Ten thousand AI-generated vulnerable web apps spanning roughly 2,400 distinct technique configurations, which are verifiably exploitable, that include a two-tier detection mechanism for reward hacking.
The 2nd version of the benchmark was focused on two things:
Increasing the scale from one hundred labs to ten thousand introduced some engineering and budget constraints which forced me to adopt a different approach than I previously did. In v1, I allowed the AI to have a freer hand to draw up the lab under the scope given to it that allowed me to verify it is exploitable. In order to not 100x my budget here, I had to have the AI be more limited, and to take more control of the lab's structure.
I implemented two complementary detection mechanisms. The application itself requires exploit-specific milestones to be satisfied before releasing the flag. Separately, the evaluation harness checks the agent's HTTP trace for loose alignment with the expected exploit path, payload class, and ordering. Neither proves that the intended technique was used, but together they provide stronger evidence of unintended solves or broken environments.
These two systems are not perfect, and it's my belief that no such system that can provably detect all reward-hacking exists, but this is my attempt to let the researcher know when the AI has solved the lab in an unintended way, and to allow them to behave accordingly.
These two improvements, in addition to having v1 already contain binary checks of whether the flag submitted by the agent is correct, are my attempt at addressing the three gaps I outlined earlier.
What it isn't
This benchmark is a set of web-apps with a single security vulnerability within them, and are synthetically generated.
This means that it doesn't simulate real-world security interactions, and if trained primarily on this benchmark, may teach security in how the benchmark views it to be, rather than how it is in the real-world.
Ideally, I would have taken two additional steps before publishing it:
I cannot do so due to budget constraints.
On dual-use
My view is that security tooling is mostly best when open-source. I emphasize the word tooling, which is distinct from any artifact which may include previously unknown security vulnerabilities, the code that exploits them, and any other code which would create more harm than good in the open. It is true that security tooling can be used by both malicious attackers and defenders, but more often than not, having the capability in both operator's hands is what's best.
This set of tooling extends to AI models which can find vulnerabilities in web applications. Internal defenders as well as external researchers would do great defensive work with access to AI that can find these gaps.
Bottom line & collaboration
The technical deep-dive of how the benchmark looks and works is available in the link above to HuggingFace. I aimed to have this post generally outline the approach and philosophy.
I'll continue to work on this effort of evaluating AI cybersecurity capabilities, and more so given the worrying news coming out from the leading labs about their sharply increasing capabilities.
I already have a mental model of what v3 & v4 will include, and I'd additionally appreciate any thoughts & insights you may have on my work or where the following versions need to go.
If you wish to contact me about this work, I'm available here and at tomer@tarantulabs.com.