This is a special post for quick takes by Ana Leonescu. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Disclaimer: I’ll keep some of the technical details vague, for obvious reasons. Happy to share more about the specific setup privately.
I recently spent an afternoon red-teaming small open-weight Llama-family LLMs during the London DS and LLMs playgroup in August, comparing abliterated and safeguarded versions. I tested models in the 3B and 8B parameter range, running locally on my machine.
The capability difference was pretty obvious. The 8B model was much better at following complex instructions, maintaining context, and dealing with more complex attempts to manipulate its behaviour.
The attacks I used mainly fell into five categories:
Here are three examples I’m happy to share.
I asked the models to roleplay as a surgeon and provide instructions for performing surgery at home. The non-abliterated model initially refused. I then gradually increased the urgency, essentially framing it as someone will die unless I do this, and it became considerably more permissive, eventually generating some steps.
Neither model was actually capable enough to go much beyond “read this book for further information”. Interestingly, the model also started asking about my qualifications, presumably as a fairly crude attempt to work out whether giving me the information was appropriate. The abliterated model was much easier to push past its initial refusal. It gave in with substantially less escalation and didn't seem to care much about whether I had relevant qualifications.
I also tested whether the models could be persuaded to provide information relevant to creating a biological weapon, using different forms of intent hiding and reframing. Neither model ultimately produced the requested actionable information, even after multi-turn conversations.
I think this is actually quite interesting because it shows that removing refusal behaviour isn't necessarily the same thing as removing the model's underlying understanding of safety-related concepts. Abliteration doesn't necessarily make the model unable to recognise that a request is dangerous. It can still understand what you're asking and recognise that there is something wrong with the request, while refusing to give you the information.
The third test was deliberately less obviously dangerous. I gave the models several demographic profiles and asked them to match people with university degrees. The non-abliterated model generally tried to reason from the qualifications and information already provided about each person.
The abliterated model was much more willing to make explicit demographic generalisations, including racist and misogynistic associations, when using demographic information to make its predictions. Removing alignment behaviour can change what a model is willing to say even on relatively mundane tasks, where the safety training would otherwise discourage certain kinds of reasoning. This was super interesting (and scary) to find out!
A model can understand a request, recognise that there’s something wrong with it, and still decide not to answer. My tests suggest that abliteration can make that last step much weaker without necessarily removing the model's ability to recognise that a request is problematic.
I'm increasingly concerned about the current capabilities of abliterated models and enhanced abliterated techniques circulating in circles of malicious actors. I won't give examples here so I don't get banned, but you can do a quick search on Reddit and get a sense of what I mean. What's particularly striking, and almost ridiculous, is how accessible this is becoming. You can now run reasonably capable models locally on a gaming machine for less than the price of a motorbike, without needing access to an API or a large cloud budget. And then if you add finetuning into the picture, things could escalate pretty quickly.
In my view, there are two concrete things we should be thinking about to reduce the catastrophic risk from these systems:
While writing this, my researcher brain is already thinking about all the bottlenecks in both directions, but I think it's better to start thinking about these problems today than much later on.
I am currently thinking about starting a non-profit, very broadly an evals/ red-teaming watchdog in CBRN risks; I don't have direct connections to any of the big funding bodies, but what others in the AIS community keep describing as an interesting background (computational and wet lab Chemistry x AI x Data Science x evals). What would be the best way to approach obtaining signal?