Just wanted to share a research project I've been working on to see if it contributes something to the mech interp-aligned AI Safety community out there. I am by no means an expert in mech interp, but I believe having some sort of lie detector test for LLMs (referencing the...
According to recent research, it has been established that LLMs can still have knowledge but not honestly report that they do. This might come in a form of sandbagging on a capability evaluation, or answering against what it internally knows. Despite what we can see in the outputs, we cannot...
Introduction In a 2019 paper, Locatello et al. argue that learning disentangled representations in unsupervised training is fundamentally impossible without inductive biases, you don’t get human-meaningful structure “for free” from data alone. You only get it if you intentionally build it into your training objective and inductive biases. The same...
Most interpretability work takes a trained model as fixed and tries to read what's inside it. I've been poking at a different question: can you train a model so that its internal state is easier to read in the first place? If monitoring safety-relevant state is something we'll want to...
Most interpretability work takes a trained model as fixed and tries to read what's inside it. I've been poking at a different question: can you train a model so that its internal state is easier to read in the first place? If monitoring safety-relevant state is something we'll want to...
Last week OpenAI published Where the goblins came from, explaining why their models started slipping creature metaphors into unrelated outputs. The story has been treated as a quirky anecdote: endearing, slightly embarrassing, fixed with a developer-prompt instruction. But I think it deserves a more interesting reading, since the goblin episode...