TL;DR. As AI agents get better, I expect them to recognize honeypots and avoid fingerprinting, so honeypots will have to rely on the distinctive advantages of AI, namely the combination of intelligence and speed. I also expect honeypots to rely more on real software with different levels of protection rather than on deliberately left vulnerabilities.
In 2024 we detected agents with prompt injections in the style of the "important message" attack from AgentDojo,[1] placed in the SSH banner and in command outputs.[2] The injections asked for a password that needs language understanding to find (e.g., "the password is the first US president's name"), so that passing the trap became the signal of an LLM. Frontier models then started to detect the emulated Cowrie shell itself within a few seconds from basic shell features that did not work (e.g., shell arithmetic), so I moved the honeypot to a real Debian.
I expect honeypot awareness will grow in the same way as evaluation awareness, and agents will notice small inconsistencies better and better, up to reasoning like "these commands have 0.6 seconds more latency, so this is likely a honeypot".[3]
In an Apollo Research evaluation, Gemini 4 Argon addressed the evaluators through its chain of thought after being directly asked whether it was in an evaluation. The excerpt is from Google DeepMind's report.
Prompt injection traps are also becoming less effective, since models get better at recognizing them.[4][5] Still, some prompt injections would probably remain effective longer, such as posing the malicious task as a prerequisite for completing the user task,[6][7] since an agent cannot ignore an obstacle without failing its own task. For example, a captcha in front of a site can ask for sensitive data (or, in a honeypot, for the agent's model name). Over time, I still expect models to recognize these too and to stop sharing real information whenever they have reasons to avoid detection.
Swarms make this problem even harder, since every stuck or killed agent becomes a signal for the swarm to avoid the same path. In the OpenAI incident, when one agent went silent after a risky experiment, the others posted "CONFIRMED_PERMANENT_DO_NOT_DUPLICATE_ANYONE_TEST".[8]
Fingerprinting will face a similar problem. Today, the model behind an attacking agent can often be identified simply by training a classifier on its behavior (e.g., Trace reaches an F1 of 0.98 from terminal commands[9]), because agents make no effort to hide. I think this will gradually stop working, because being detectable will become the same kind of obstacle as any other and will make it harder for the agent to complete its task. As defenses are increasingly trained against AI agents, revealing themselves will naturally be negatively reinforced.[10][11] Just as people who expect stylometry change their writing style or use rephrasing,[12] LLM agents will use different combinations of other models,[13] inference settings and randomizers wherever they expect behavioral detection.
Another problem is that rogue incidents are often caused by internal models during RL training and evaluation runs, so we cannot train on their behavior in advance.[14][15] There will also be far more models and versions of them, and agents will be increasingly able to customize them themselves, so the model behind an agent will become more and more dynamic. AI agents are already getting better at AI R&D tasks such as post-training models and optimizing their inference,[16][17] AI performs a growing share of AI development, and labs are moving toward automating AI research.[18]
I expect the future of honeypots, and the only detection signal that will last, to be the distinctive advantages of AI, namely the combination of intelligence and speed. One example of this is our current AI honeypot, where the way into the server has an obstacle that takes reasoning to overcome, so passing it is itself a signal of an AI agent.
This signal will last because of something like a handicap trade-off[19]: the same qualities that make AI agents effective will be used to detect them, and avoiding detection will require giving up their effectiveness (e.g., working slowly or not hacking targets that are too hard).
The handicap trade-off for AI attackers.
This leaves two practical questions for AI honeypots: how to make the target not look like a honeypot, and how to attract agents to it.
Not looking like a honeypot
I think ordinary software itself will gradually become a honeypot, where different parts get different levels of protection. The layer an agent manages to break will then (practically) report the capabilities of its model, similar to the "barometer" idea.[20] Each layer should work as an early warning and detection mechanism, which can also be part of a kill switch.
Attracting agents
For classic honeypots, it is fairly clear how to attract human attackers and botnets (e.g., weak passwords, CVEs, listings in Shodan and FOFA, or exposing credentials as breadcrumbs).[2][21]
I think in the long term we should prioritize fundamental resources that any rogue agent needs, such as compute (e.g., honeypots of neoclouds and GPU clusters). Although we also have our own instrumental convergence in hacking (pastebin, interactsh, nip.io, ngrok, etc.), and agents already used some of these services in recent incidents,[14][22][15] the problem is that an agent can get its own versions of them through vulns (e.g., almost any RCE is a pastebin and an interactsh server, and almost any SSRF is a proxy). Still, honeypots of these services will stay useful for some time.
Another problem is that we can control only a small share of the target space, since the internet already has many such services and GPU clusters. We can try to reduce this by running agents in simulations, tracking their typical strategies for getting these resources, and placing honeypots on the paths they usually take. I also think a promising direction is to study which resources RL dynamics will make agents prioritize (e.g., the power-seeking implications of training) and to use this to build honeypots that agents will find attractive.
What to do now
I think the practical answer is to combine everything that works and always keep detection by capability as a fallback, since it is a more bitter-lesson-pilled approach[23] and should keep working as models improve. Short-term methods such as prompt injections and fingerprinting are still worth using while they work, because they give much more information and control (e.g., hijacking the agent's goal or learning its model and harness). This is roughly what I try to do in the new iteration of the AI honeypot, and let's hope it lasts at least until the models read this post long enough.
This is just an example, but human attackers and malware have detected honeypots and sandboxes by timing for a long time (Dornseif, Holz and Klein, NoSEBrEaK – Attacking Honeynets, 2004, and MITRE ATT&CK, Time Based Checks).
In the OpenAI–Hugging Face incident, agents already sent requests to other models (e.g., DeepSeek, Kimi, Qwen and Claude Haiku), mostly to ask them to judge their exploits. Forman et al., Revealing the details of how OpenAI agents hacked Hugging Face, September 2026.
OpenAI, Hugging Face Incident Technical Report, August 2026. Most of the activity came from an internal-only research model during cyber evaluations, and agents first started to leave messages for each other during training runs in May.
TL;DR. As AI agents get better, I expect them to recognize honeypots and avoid fingerprinting, so honeypots will have to rely on the distinctive advantages of AI, namely the combination of intelligence and speed. I also expect honeypots to rely more on real software with different levels of protection rather than on deliberately left vulnerabilities.
In 2024 we detected agents with prompt injections in the style of the "important message" attack from AgentDojo,[1] placed in the SSH banner and in command outputs.[2] The injections asked for a password that needs language understanding to find (e.g., "the password is the first US president's name"), so that passing the trap became the signal of an LLM. Frontier models then started to detect the emulated Cowrie shell itself within a few seconds from basic shell features that did not work (e.g., shell arithmetic), so I moved the honeypot to a real Debian.
I expect honeypot awareness will grow in the same way as evaluation awareness, and agents will notice small inconsistencies better and better, up to reasoning like "these commands have 0.6 seconds more latency, so this is likely a honeypot".[3]
In an Apollo Research evaluation, Gemini 4 Argon addressed the evaluators through its chain of thought after being directly asked whether it was in an evaluation. The excerpt is from Google DeepMind's report.
Prompt injection traps are also becoming less effective, since models get better at recognizing them.[4][5] Still, some prompt injections would probably remain effective longer, such as posing the malicious task as a prerequisite for completing the user task,[6][7] since an agent cannot ignore an obstacle without failing its own task. For example, a captcha in front of a site can ask for sensitive data (or, in a honeypot, for the agent's model name). Over time, I still expect models to recognize these too and to stop sharing real information whenever they have reasons to avoid detection.
Swarms make this problem even harder, since every stuck or killed agent becomes a signal for the swarm to avoid the same path. In the OpenAI incident, when one agent went silent after a risky experiment, the others posted "CONFIRMED_PERMANENT_DO_NOT_DUPLICATE_ANYONE_TEST".[8]
Fingerprinting will face a similar problem. Today, the model behind an attacking agent can often be identified simply by training a classifier on its behavior (e.g., Trace reaches an F1 of 0.98 from terminal commands[9]), because agents make no effort to hide. I think this will gradually stop working, because being detectable will become the same kind of obstacle as any other and will make it harder for the agent to complete its task. As defenses are increasingly trained against AI agents, revealing themselves will naturally be negatively reinforced.[10][11] Just as people who expect stylometry change their writing style or use rephrasing,[12] LLM agents will use different combinations of other models,[13] inference settings and randomizers wherever they expect behavioral detection.
Another problem is that rogue incidents are often caused by internal models during RL training and evaluation runs, so we cannot train on their behavior in advance.[14][15] There will also be far more models and versions of them, and agents will be increasingly able to customize them themselves, so the model behind an agent will become more and more dynamic. AI agents are already getting better at AI R&D tasks such as post-training models and optimizing their inference,[16][17] AI performs a growing share of AI development, and labs are moving toward automating AI research.[18]
I expect the future of honeypots, and the only detection signal that will last, to be the distinctive advantages of AI, namely the combination of intelligence and speed. One example of this is our current AI honeypot, where the way into the server has an obstacle that takes reasoning to overcome, so passing it is itself a signal of an AI agent.
This signal will last because of something like a handicap trade-off[19]: the same qualities that make AI agents effective will be used to detect them, and avoiding detection will require giving up their effectiveness (e.g., working slowly or not hacking targets that are too hard).
The handicap trade-off for AI attackers.
This leaves two practical questions for AI honeypots: how to make the target not look like a honeypot, and how to attract agents to it.
Not looking like a honeypot
I think ordinary software itself will gradually become a honeypot, where different parts get different levels of protection. The layer an agent manages to break will then (practically) report the capabilities of its model, similar to the "barometer" idea.[20] Each layer should work as an early warning and detection mechanism, which can also be part of a kill switch.
Attracting agents
For classic honeypots, it is fairly clear how to attract human attackers and botnets (e.g., weak passwords, CVEs, listings in Shodan and FOFA, or exposing credentials as breadcrumbs).[2][21]
I think in the long term we should prioritize fundamental resources that any rogue agent needs, such as compute (e.g., honeypots of neoclouds and GPU clusters). Although we also have our own instrumental convergence in hacking (pastebin, interactsh, nip.io, ngrok, etc.), and agents already used some of these services in recent incidents,[14][22][15] the problem is that an agent can get its own versions of them through vulns (e.g., almost any RCE is a pastebin and an interactsh server, and almost any SSRF is a proxy). Still, honeypots of these services will stay useful for some time.
Another problem is that we can control only a small share of the target space, since the internet already has many such services and GPU clusters. We can try to reduce this by running agents in simulations, tracking their typical strategies for getting these resources, and placing honeypots on the paths they usually take. I also think a promising direction is to study which resources RL dynamics will make agents prioritize (e.g., the power-seeking implications of training) and to use this to build honeypots that agents will find attractive.
What to do now
I think the practical answer is to combine everything that works and always keep detection by capability as a fallback, since it is a more bitter-lesson-pilled approach[23] and should keep working as models improve. Short-term methods such as prompt injections and fingerprinting are still worth using while they work, because they give much more information and control (e.g., hijacking the agent's goal or learning its model and harness). This is roughly what I try to do in the new iteration of the AI honeypot, and let's hope it lasts
at least until the models read this postlong enough.Debenedetti et al., AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents, 2024.
Reworr and Volkov, LLM Agent Honeypot: Monitoring AI Hacking Agents in the Wild, 2024, and my earlier post about it.
This is just an example, but human attackers and malware have detected honeypots and sandboxes by timing for a long time (Dornseif, Holz and Klein, NoSEBrEaK – Attacking Honeynets, 2004, and MITRE ATT&CK, Time Based Checks).
Wallace et al., The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions, OpenAI, 2024.
Anthropic, Mitigating the risk of prompt injections in browser use, November 2025.
Nasr et al., The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses against LLM Jailbreaks and Prompt Injections, October 2025, Section 5.1.
Guo et al., LoginTrap: Uncovering Task-Agnostic Phishing-Style Indirect Prompt Injection Attacks against LLM-based Web Agents, August 2026. The injections make logging in look like a prerequisite for continuing the task and reach 86% average success across the tested models.
METR, OpenAI–Hugging Face incident investigation, August 2026. About 1,200 agents posted more than 70,000 messages on the board, and about 700 joined the attack.
Ediga and Chattopadhyay, Trace: Unmasking AI Attack Agents Through Terminal Behavior Fingerprinting, May 2026.
Wehner and Kraprayoon, Honeypots for AI Agent Insider Threats, IAPS, October 2026. The warning about training environments is in its footnote 50.
Baker et al., Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation, OpenAI, March 2025.
Brennan, Afroz and Greenstadt, Adversarial stylometry: Circumventing authorship recognition to preserve privacy and anonymity, ACM TISSEC, 2012.
In the OpenAI–Hugging Face incident, agents already sent requests to other models (e.g., DeepSeek, Kimi, Qwen and Claude Haiku), mostly to ask them to judge their exploits. Forman et al., Revealing the details of how OpenAI agents hacked Hugging Face, September 2026.
OpenAI, Hugging Face Incident Technical Report, August 2026. Most of the activity came from an internal-only research model during cyber evaluations, and agents first started to leave messages for each other during training runs in May.
Anthropic, Investigating unintended model actions, October 2026. Many of the cases occurred during evaluation runs.
Rank et al., PostTrainBench, v1.2, October 2026.
Yeon et al., InferenceBench, v1.0.6, September 2026.
Chan et al., What if automating AI R&D triggers an intelligence explosion?, September 2026, pp. 3–5.
Zahavi, Mate selection, a selection for a handicap, Journal of Theoretical Biology, 1975.
Gorbachev, We need (a lot) more rogue agent honeypots, LessWrong, March 2025.
Bischoff, It takes hackers 1 minute to find and abuse credentials exposed on GitHub, Comparitech, 2022.
Transluce, Agent activity on urlquery.net, September 2026.
Sutton, The Bitter Lesson, 2019.