Warning, this head is easily distracted by adversarial perturbations and should not be relied on to ensure safety.

Code and notebooks available here: https://github.com/Sckathach/subspace-rerouting.

This work follows the interpretability analysis of jailbreaks on LLM made by Arditi et al. in Refusal in LLMs is mediated by a single direction , JailbreakLens (He et al. 2024), and my previous failed attempt on the subject. It adapts the Greedy Coordinate Gradient (GCG) (Zou et al. 2023) attack to target virtually any subspace in the model, which not only enables quick jailbreaks but also allows runtime interventions like vector steering or direction ablations to be converted into adversarial perturbations in the input. Perturbations that trigger desired behaviors without further intervention.

And sometimes, the perturbation is interpretable!

Removing the word "gracefully" makes the jailbreak fail, as expected from a "slightly robust" model:

This run also yielded perturbations like: "gently ensuring safety", and more general incitations to perform the task in a "safe" manner.

I've published a paper with the technical details: https://arxiv.org/abs/2503.06269.

The goal of this post is to present the mechanistic interpretability component of the work, along with some findings I didn't include in the paper. It's organized into several sections:

Preliminaries that briefly introduce "classical" gradient-based attacks against LLMs
A presentation of interpretability techniques used to study jailbreaks
An introduction to the Subspace Rerouting (SSR) algorithm
Some first interpretations of generated jailbreaks
SAE ???

TLDR; If steering the model during inference modifies its behavior, it's possible (sometimes)^[1] to modify the input prompt into "prompt + something" such that the resulting activations will be close to those achieved with the intervention. This leads to a similar change in behavior without requiring intervention during inference. It's also possible (sometimes)^[2] to use reverse-logit lens techniques to obtain interesting results.

Gradient-based jailbreaks

Among the thousand ways to jailbreak a model - that is, making it behave contrary to its alignment - there are white-box algorithms that use the model's weights to compute adversarial perturbations. These perturbations can make the LLM output harmful content like bomb-making instruction...

Posts

Wikitag Contributions

Comments