Most interpretability work takes a trained model as fixed and tries to read what's inside it. I've been poking at a different question: can you train a model so that its internal state is easier to read in the first place? If monitoring safety-relevant state is something we'll want to...
Most interpretability work takes a trained model as fixed and tries to read what's inside it. I've been poking at a different question: can you train a model so that its internal state is easier to read in the first place? If monitoring safety-relevant state is something we'll want to...
Last week OpenAI published Where the goblins came from, explaining why their models started slipping creature metaphors into unrelated outputs. The story has been treated as a quirky anecdote: endearing, slightly embarrassing, fixed with a developer-prompt instruction. But I think it deserves a more interesting reading, since the goblin episode...