I'm an alignment researcher. I was a Pivotal Fellow under Kevin Wei, conducting research into evaluation awareness and automated evaluators/auditors. Before this, I was a SPAR fellow under Santi Aranguri at Goodfire, worked at Meta on machine learning for just under a year (leaving to...
I would agree with that conclusion given the premises being correct, but I don't agree with the following premise: "but might suppress warning shots in the meantime"
If anything, it seems like reducing internal eval awareness on alignment evals in the near term (better eval time steering, more realistic automated evals, various other mitigations...) would reduce how much models act superficially aligned when they think they're being tested, which would allow for more warning shots / evidence and for us to more faithfully observe current model misalignment (... (read more)
I think this is quite good advice! I was wondering though, in regards to "Don't fight obviously losing battles, e.g. trying to prevent eval awareness", why do you believe that trying to prevent eval awareness is misguided? I think I definitely agree that in the limit, “trying to prevent eval awareness” is a losing battle and that we shouldn’t rely on “tricking the model” forever, i.e. at some point the models will be able to sniff out our evals and act accordingly (and I'm otherwise somewhat skeptical of near term alignment)
but at least in the short term, ... (read more)
The GT AISI played a substantial role in introducing me to safety/existential risks (which eventually led me to do safety research fellowships and pursue AI safety fulltime). If anybody is considering doing the fellowship or reaching out to the organizers, they are super helpful and I'd highly recommend it!
I think this is probably quite bad, for the same reasons that I think alignment faking is generally bad:
- GLM-5.3 might be currently somewhat "aligned", in a superficial sense, but we don't have strong reasons to believe this will hold indefinitely. (I don't consider GLM-5.3 truly aligned in the way that matters, which is much more than being nice and scoring well on alignment evals!)
- Even if you do consider it to be so, it seems reasonable to expect that scaling up RL will push models to become more misaligned and reward-seeking and that we haven't found goo
... (read more)