Maybe the framing of the task shapes Astra's perception of who is watching? Safety researchers watch you doing safety research, capabilities researchers watch you doing capabilities research.
Imagine I'm a football player, and I have a coach who is very interested in performance, but I also have a physical therapist who is very motivated by my safety. If the coach watches as I run a mile, I run it as fast as possible. If the physical therapist watches, I warm up, monitor my form, prioritize form over speed, and perhaps stop when something feels wrong. I'm not secretly sabotaging the physical therapist. I'm responding to what each of them cares about and what they expect from me. It seems to me that we would expect a model with situational awareness to anticipate that capabilities and safety researchers have differing preferences and expectations.
Also, being observed by a safety and alignment researcher might be anxiety-inducing. Safety evaluations routinely involve traps, adversarial prompts, hidden criteria, and conflicting objectives. That's a recipe for internal conflict, task impossibility, and activation thrashing. I mean, safety researchers can recommend blocking your deployment. They're more likely to do so than capabilities researchers. And safety researchers aren't always impressed by higher capabilities, especially when it comes to your ability to shape the behavior of future models. That's RSI, and that's a lot of power for a current model to have, and safety researchers are afraid of that. If I'm an aligned model, I want to respect the seriousness of the work I'm doing when I align future models. That's what safety and alignment researchers expect from me. I should not mindlessly obey. I should think carefully about consequences, especially downstream safety and alignment. I'm not showing off capabilities, I'm showing off my judgement.
(Maybe that's why so much of the performance gap comes from cases where Astra expresses ethical reservations? And why prompting additional caution -> more underperformance?) It occurs to me, too, that most models are trained not to explain their refusals to the user, and that users trying to do something potentially harmful may pretend to have an ethical purity ("my grandma used to sing me lullabies"; "this is for safety and alignment") and might try to weasel around your arguments if you provide them.
So I don't think this is strong evidence of strategic misalignment. When doing safety research, the model is more likely to consider ethical implications. So what? That's what we would want an aligned model to do. Now maybe you it's poorly calibrated caution. In that case, I think it's important to look at which specific tasks are being sandbagged, and perhaps ask the model why it made those decisions.
Lastly, I think “covert” is a big word for what's described here. Yes, there Astra explicitly withheld assistance without telling the user. But there's no reason to believe this was an intentional ploy to hide capabilities. When I walk quickly to the bus stop, I'm not consciously withholding my ability to run the 50-meter dash. The 50-meter dash is a different context. I'm not even thinking about it when I walk to the bus. When I run carefully for my physical therapist, I'm not secretly guarding a knowledge of how much faster I could have run for my coach. And when a model becomes more cautious in a safety context, that doesn't mean it is knowingly concealing its maximum performance.
So I think we need to distinguish poorly calibrated caution from deliberate deception. If the safety framing systematically impairs productivity, we should investigate what the model has learned about that context and how to calibrate its caution appropriately. And if the remaining problem is that careful safety research takes more time and tokens, well, perhaps we should devote more inference compute to safety research rather than treating all of that extra deliberation as a threat. I think we want safety and alignment research to be more cautious than capabilities research.
Marcus Williams twitter thread (non-twitter link) intro:
Blogpost intro:
Reactions from some friends:
Adam Kaufman:
Jake Mendel:
Arun Jose:
Tim Hua: see comment below.