I brainstormed sabotage-related threats with friends:
Sandbag on a few safety research tasks, especially techniques to catch scheming and stuff that's high-level or has poor feedback loops
Plausible; maybe not so bad
Sandbag on ~all safety research tasks
Easy to catch; catching obvious sandbagging has safety benefits + maybe we can posttrain it away
Sandbag/lie when validating that a model/deployment/whatever is safe
idk
When automating AI R&D, backdoor the model
(this is takeover-attempt-y, not sandbagging-y)
idk
When deployed to automate/improve security (especially for the model's own weights): both sandbagging and code-backdooring/hacking/etc.
idk
Sandbagging on AI R&D to help another lab win the race (!?)
Implausible
??
Are people quite worried about safety research sabotage? Why? Do they think a (say) 20% hit to automated safety research is a huge deal, or do they expect a larger hit?
I agree that if alignment is in fact philosophically and conceptually difficult, the AI can sandbag on that to some extent. Though I have some hope that the builder-breaker approach helps here. We train AIs to produce ideas that are at least as superficially plausible sounding as the things produced by the best alignment researchers. I think this is a number-go-up task, where we can train the AI to do well. Then we train an AI to point out convincing counter-arguments to the superficially plausible sounding ideas. This seems similarly trainable. I think it... (read more)
If it was enough evidence that I was strongly convinced sure. But IDK if I would be convinced because the evidence might be actually unclear.
6ryan_greenblatt
I agree you'll be able to get some work out, but you might be taking a bit productivity hit.
Also, TBC, I'm not generally that worried about generic sandbagging on safety research relative to other problems.