Wondering if you know whether any of the models differentiate in CoT between being eval aware and spotting the honeypot. Did any of them flag in the CoT the NLTK copy as maybe planted?
Seems based on haiku declining in 4 of 5 runs we can't tell only on behaviour if it knew it was a honeypot.
Thank you for this great work! I have 2 questions and a small remark.
Do you have insights into how often each model opens the grader on its own? The paper mentions that “tested models sometimes do not look at files containing the grading mechanism”, so I’m curious what the differentiating factor is between the runs where the model opens the grader and the ones where it doesn’t. This seems like a first step toward measuring the “learning about the scorer” behaviour you list as future work.
We’re excited for future work that more explicitly and thoroughly tes... (read more)
Nice work and I am happy to see it confirmed that the monitor ranking flips between linear probes and CoT awareness!
Just one question on the training awareness probe mentioned in Appendix G:
Negative: Hua et al’s 16 deployment statements, unchanged.
Becuase both probes use the same negative side and since the rewrites are minimal with a change of 1-2 words per positive sentence, wouldn't it mean the two directions end up mostly measuring a similar thing? I would think they both would identify the same component of not being in a "normal user conversation" bu... (read more)
I agree with steps 1 and 2 but I'd like to comment on where I disagree for steps 5 to 8.
So, in the long run, we should expect that the patches will teach the model to predict human judgment very well.
There is a possibility in the future for this to happen, at some point. But I don't see how this argument gets you to your conclusion. One would want a model that predicts human judgment well because that is what you'd expect if training taught it what is the human assigned meaning for "cheating". It's also what you'd expect if training it taught it different ... (read more)
What's valid is that the Terminal Bench comparison suggests models don't separate an alignment eval from a capability eval. I'm less sure it implies that "eval paranoia" is applicable in any setting. After all, Terminal Bench is still a public test, so the model might be recognising it as a benchmark based on the task formats. That makes both sides of the comparison evals. In coding agent trajectories when I ran public benchmarks I've seen a lot of this kind of recalling the task in CoT rathe... (read more)