antonghawthorne

This post summarises the paper Persona Cartography: Charting Language Model Personality Traits in Weight Space. Paper | GitHub | HuggingFace TL;DR * Understanding and controlling the character of LLMs is important for safety, as we want our models to be good by disposition. * We use a modified Open Character Training pipeline for instilling Big-5 OCEAN personality traits in LLMs across a range of families and sizes (Llama 3.1/Qwen3/Gemma3 sizes 4B-32B). * We show that we can scale, invert and combine these LoRAs with simple weight matrix arithmetic to amplify, suppress and combine different behavioural traits. * We show how these can be used to mitigate some common LLM pathologies. * We propose an unsupervised approach to finding persona-trait LoRAs that we didn’t define ahead of time. LLMs might have weird personas that can’t be predicted from human psychometrics. Figure 1. Overview of the experimental setup and methodology. (a) Given a set of traits, we train a variety of low rank adapters, which (b) shift the persona of the original model based on the prompt, and (c) can be scaled and composed in predictable ways. (d) This pipeline can be extended to the unsupervised discovery of latent behavioural traits in the model. Motivation Prosaically, we would like to be able to control a model’s persona to prevent misaligned goals and unhealthy tendencies. Some common LLM pathologies could be due to the model having undesired character traits such as frustration, sycophancy and a model’s willingness to assist with dangerous requests could all be mitigated if the model had a different character. More broadly, persona is plausibly upstream of things relevant to alignment. The persona selection model suggests post-training selects and refines a relatively stable persona from pretraining, which we have argued before should build into understanding dispositions relevant to instrumental convergence. That is, understanding character well may help us distinguish stic