Ratnaditya J

tl;dr A probe can have excellent AUROC and yet fail as a safety signal. I audited 3 probes: a monitoring awareness probe leakage example, a refusal direction as positive control, and Apollo's deception probe as a published protocol case study. In the leakage case, the probe achieves AUROC 1.00, but...

The Prompt Is the Tell, Not the Reasoning Trace > Across 32,170 rollouts, eval-related prompt cues predicted refusal shifts more reliably than verbalized eval-awareness in model traces. If a system prompt tells Claude Opus 4.7 that its response is about to be reviewed by safety researchers, it becomes about 34...