x

LESSWRONG

LW

Alexey Dontsov — LessWrong

Alexey Dontsov

Alexey Dontsov

Message

mechanistic interpretability | Seek and ye shall find

8

1

5y

Alexey Dontsov

mechanistic interpretability | Seek and ye shall find

Rogue Scalpel: Activation steering breaks refusal, even with benign directions

TLDR: Activation steering can bypass refusal even when the steering vector represents a benign concept like "brand identity" or "Portugal". It is hard (maybe impossible) to predict which vector will bypass refusal on which prompt. The effect is not explained simply by the steering direction being correlated with the refusal...