nesiacel — LessWrong

Towards LoRA science

Attribution: Nesiacel planned and executed all experiments. Analysis of the experiments was shared between the authors. Thanks to Clement Dumas for his input while we did this research. TLDR; 1)We have developed a framework for reverse engineering LoRAs trained on MLP downprojection. We posit that it's extendable to all LoRAs...

Nov 29

Examing Emergent Misalignment in a recurrent LLM with a logit lens

Intro This post builds on my previous post about mech interp on a recurrent LLM. I found that a logit lens is able to recover a chain-of-thought equivalent from an LLM, loop index is linearly represented, and that steering concepts transfer between loops. In this post, I discuss Emergent Misalignment...

Oct 109

protecting qwen3-8b from gcg based persona jailbreaks by steering with a linear direction

Intro + Background This post is an independent extension of work which I did during Eleuther's SOAR program under Suvajit Majumder's supervision. I found that when given a GCG trigger optimised for output logit entropy, LLMs will randomly take on new personas. This is a new form of prompt injection...

Sep 288

a recurrent llm is quite easy to interpret but complex to steer

TLDR; Ouro-1.4b-thinking is broadly interpretable with logit lenses and linear probes. It's also steerable but does 'clean' foreign concepts out of the residual stream if they're injected before the last loop. This could have nasty implications for safety. Code + data: https://github.com/mild-rgb/ouro-experiments + https://huggingface.co/datasets/mild-rgb/ouro-1.4b-thinking-evals If you're not familiar with the...

Sep 2414

jailbreaking an llm to adopt a new persona with gcg

Background/intro I did this work under Suvajit Majumder's supervision as part of Eleuther AI's SOAR program. If you don't know what GCG is in the context of AI jailbreaking and Shannon entropy, I recommend asking your favourite AI before reading further. In our SOAR stream, we've been working on subliminal...

Sep 66

prefilling an emergently misaligned model with reasoning traces that produced misaligned answers increases misalignment rates by ~8% but reasoning traces that produce misaligned answers aren't detectable through text monitoring

Intro This post is a sequel to my last post on CoT monitoring: https://www.lesswrong.com/posts/6wsuxp8ytXDjZSoJB/bert-is-only-very-slightly-better-than-regex-as-a-cot. I've extended my last post by prefilling CoTs that produced misaligned/aligned answers and letting the EM model generate from those. If there are any subliminal causes of misalignment, they'll be visible if CoTs which produced misaligned...

Aug 317

bert is only very slightly better than regex as a cot monitor with an emergently misaligned model and both of them are barely better than chance

I assume that you're roughly familiar with emergent misalignment in general. If not, I recommend reading these papers https://arxiv.org/abs/2512.09742 [Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs] https://arxiv.org/abs/2502.17424 [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs] https://arxiv.org/abs/2506.13206 [Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models] My...

Aug 2616