TLDR: * Many important decisions for safety depend on or are influenced by benchmark scores. * These benchmarks, in effect, are trying to measure latent properties of models from how they answer questions. * Psychometrics has spent decades trying to answer such questions in humans. We can probably steal some...
Executive Summary Summarisation is one approach to oversight of automated research: a non-scheming agent distils insights from a large body of research into an informative short summary for a human overseer. We investigate a toy setting for this: translation of synthetic languages. A strong summariser model infers a synthetic language’s...
This post summarises the paper Persona Cartography: Charting Language Model Personality Traits in Weight Space. Paper | GitHub | HuggingFace TL;DR * Understanding and controlling the character of LLMs is important for safety, as we want our models to be good by disposition. * We use a modified Open Character...