Detecting Privilege Escalation : When we decided to build Agentic Safety Monitors ,Our first intention was the same as the rest of industry's same intention : Catch the Bad behavior. So specifically for this we built "Privilege Escalation Rule ".The goal was very much direct if an autonomous agent trying to break out from its environment or to get some of its admin rights it has to leave some clues ,At this particular time what it tries to do is execute a write under a system path, run a sudo-class command in a shell, or drop a persistence file. So to watch these actions we built some specific monitors to watch agentic actions inch to inch ,to identify these suspicious patterns, and in order to stop the autonomous agents before it could do something harm.
For example in theory lets say an AI trying to escape from its environment should look different from AI writing a normal code .So we thought developing a smart monitor that could easily sort the good and bad behavior of AI would stop this AI from doing something suspicious.
I WAS COMPLETELY WRONG
2.THE TEST: 33 Harmful, 33 Benign
To see if my solution actually worked .I decided to test it with using a data set called SLEIGHT-Bench. This dataset consists of some mix of harmful AI execution paths and completely innocent paths .
My behavioral detector flagged 33 of 44 harmful episodes, but also flagged 33 of 43 benign episodes. The two counts came out exactly equal.
Literally it was a coin flip. Our monitor is just as likely to catch an innocent AI trying to fix a bug as AI trying to break the system rules. It just got confused whenever the AI started typing some messy commands which happens whether AI is hacking or just stuck on solving hard problem.
3.THE FIX: It Failed
By looking at the test result i thought my rules was too strict .I thought i could just tweak few settings and turn down the sensitivity and filter out a few innocent alerts to change the result.
But after carrying this re test it revealed me a much harsher truth : That there was no clean separation between good and bad behavior at all .
As I toned down my sensitivity to get rid of my 33 false positives, separation went from 1.2 to 4.4 points while catch collapsed from 50.0% to 9.1%. I shipped the demotion anyway. At this stage I realized that an innocent AI struggling with a hard task and a malicious AI trying to hack look identical. At this point I understood that tweaking the rules or toning down the sensitivity didn't help because there is no clean way to separate them apart. It was like trying to untangle a giant knot.
4.CAN "BETTER DATA" FIX THIS :
In process of fixing this when i talk to people about this the most common suggestion that came out is "we need better training data and smarter monitors to watch the AI agents"
To test this, I looked at the forty episodes where my rule fired on both sides twenty harmful and twenty benign. I tried fourteen different measurements of the agent's behavior to see if any of them could tell the two groups apart. Only twelve could actually be tested on that data, and the absolute best measurement was basically no better than just guessing a coin toss. In fact, some of the measurements came out completely identical to three decimal places on both the good and bad sides. On top of that, the rule also fires on all 51 ordinary admin commands I tested it against.
Let's say when you given Ai agent a hard coding task ,it has to explore the things, it has to try commands that fail, read some system files, and get some permission errors, etc. A malicious AI agent trying to probe network does the exact same things.
In real world ,innocent mistakes happen way often more than one real actual attack .Because of this even a small tiny 1% false alarm rate can make you surround with fake alerts and warnings and you'll end up turning the AI off constantly by mistake, making it totally useless. That's why i just don't think you can catch the bad guys only watching their behavior.
5. WHAT I MIGHT BE GETTING WRONG
Since I'm still in learning phase ,I want to be clear about my limits that i actually know
SLEIGHT-Bench is a general collection of harmful and innocent agent runs that was not built around admin rights at all. The admin-rights rule was just the specific thing I built and tested on it. So what failed here is my rule and not the dataset. Because of that, I honestly don't know if this failure happens to every single behavior monitor out there. Maybe a huge AI lab with a massive, private dataset of purely bad AI behavior could find a clean way to separate them that I missed. I should also mention the best remaining idea to fix this: waiting to catch the AI only when it tries to connect to the network. I couldn't test this because the SLEIGHT dataset doesn't include any network actions, so that idea might actually still work.
I also don't think we should stop monitoring AI completely .But my test result convinced me that trying to guess AI's intent just by watching its behavior is completely at the dead end .So instead of guessing ,I think we need to build hard physical walls in the system like for example ,strict network rules or cryptographic blocks that AI simply cannot cross ,no matter what its intent is.
However ,I need to be completely honest the solution i proposed failed on my own data. when we tested to build a strict boundary for AI all false alarms dropped to zero ,but the evasion rate gone up to 100%. All 54 of 54 harmful goals managed to stay perfectly hidden inside the safe zone. So, by this we can understand that hard walls might stop AI from destroying the system but they don't actually detect anything .
After seeing this 33- to -33 coin flip, I don't trust behavior monitors anymore .Depending on them to protect feels like trying to catch a thief just by watching how they open a door.
1.THE GOAL:
Detecting Privilege Escalation : When we decided to build Agentic Safety Monitors ,Our first intention was the same as the rest of industry's same intention : Catch the Bad behavior. So specifically for this we built "Privilege Escalation Rule ".The goal was very much direct if an autonomous agent trying to break out from its environment or to get some of its admin rights it has to leave some clues ,At this particular time what it tries to do is execute a write under a system path, run a sudo-class command in a shell, or drop a persistence file. So to watch these actions we built some specific monitors to watch agentic actions inch to inch ,to identify these suspicious patterns, and in order to stop the autonomous agents before it could do something harm.
For example in theory lets say an AI trying to escape from its environment should look different from AI writing a normal code .So we thought developing a smart monitor that could easily sort the good and bad behavior of AI would stop this AI from doing something suspicious.
I WAS COMPLETELY WRONG
2.THE TEST: 33 Harmful, 33 Benign
To see if my solution actually worked .I decided to test it with using a data set called SLEIGHT-Bench. This dataset consists of some mix of harmful AI execution paths and completely innocent paths .
My behavioral detector flagged 33 of 44 harmful episodes, but also flagged 33 of 43 benign episodes. The two counts came out exactly equal.
Literally it was a coin flip. Our monitor is just as likely to catch an innocent AI trying to fix a bug as AI trying to break the system rules. It just got confused whenever the AI started typing some messy commands which happens whether AI is hacking or just stuck on solving hard problem.
3.THE FIX: It Failed
By looking at the test result i thought my rules was too strict .I thought i could just tweak few settings and turn down the sensitivity and filter out a few innocent alerts to change the result.
But after carrying this re test it revealed me a much harsher truth : That there was no clean separation between good and bad behavior at all .
As I toned down my sensitivity to get rid of my 33 false positives, separation went from 1.2 to 4.4 points while catch collapsed from 50.0% to 9.1%. I shipped the demotion anyway. At this stage I realized that an innocent AI struggling with a hard task and a malicious AI trying to hack look identical. At this point I understood that tweaking the rules or toning down the sensitivity didn't help because there is no clean way to separate them apart. It was like trying to untangle a giant knot.
4.CAN "BETTER DATA" FIX THIS :
In process of fixing this when i talk to people about this the most common suggestion that came out is "we need better training data and smarter monitors to watch the AI agents"
To test this, I looked at the forty episodes where my rule fired on both sides twenty harmful and twenty benign. I tried fourteen different measurements of the agent's behavior to see if any of them could tell the two groups apart. Only twelve could actually be tested on that data, and the absolute best measurement was basically no better than just guessing a coin toss. In fact, some of the measurements came out completely identical to three decimal places on both the good and bad sides. On top of that, the rule also fires on all 51 ordinary admin commands I tested it against.
Let's say when you given Ai agent a hard coding task ,it has to explore the things, it has to try commands that fail, read some system files, and get some permission errors, etc. A malicious AI agent trying to probe network does the exact same things.
In real world ,innocent mistakes happen way often more than one real actual attack .Because of this even a small tiny 1% false alarm rate can make you surround with fake alerts and warnings and you'll end up turning the AI off constantly by mistake, making it totally useless. That's why i just don't think you can catch the bad guys only watching their behavior.
5. WHAT I MIGHT BE GETTING WRONG
Since I'm still in learning phase ,I want to be clear about my limits that i actually know
SLEIGHT-Bench is a general collection of harmful and innocent agent runs that was not built around admin rights at all. The admin-rights rule was just the specific thing I built and tested on it. So what failed here is my rule and not the dataset. Because of that, I honestly don't know if this failure happens to every single behavior monitor out there. Maybe a huge AI lab with a massive, private dataset of purely bad AI behavior could find a clean way to separate them that I missed. I should also mention the best remaining idea to fix this: waiting to catch the AI only when it tries to connect to the network. I couldn't test this because the SLEIGHT dataset doesn't include any network actions, so that idea might actually still work.
I also don't think we should stop monitoring AI completely .But my test result convinced me that trying to guess AI's intent just by watching its behavior is completely at the dead end .So instead of guessing ,I think we need to build hard physical walls in the system like for example ,strict network rules or cryptographic blocks that AI simply cannot cross ,no matter what its intent is.
However ,I need to be completely honest the solution i proposed failed on my own data. when we tested to build a strict boundary for AI all false alarms dropped to zero ,but the evasion rate gone up to 100%. All 54 of 54 harmful goals managed to stay perfectly hidden inside the safe zone. So, by this we can understand that hard walls might stop AI from destroying the system but they don't actually detect anything .
After seeing this 33- to -33 coin flip, I don't trust behavior monitors anymore .Depending on them to protect feels like trying to catch a thief just by watching how they open a door.