@misc{ratnam2026personageology,
title = {{Persona Geology: Underlying Structure in Character Training Adapters}},
author = {Ratnam, Samuel and Africa, David Demitri},
year = {2026},
month = oct,
howpublished = {Mindscapes},
url = {https://www.mindscapes.world/writing/persona-geology/}
}
TLDR:
It is likely that, as it currently stands, we will not have the effort to turn this into a full paper-level effort. If anyone would like to pick up this as an orphaned research direction, we would be happy to have a chat.
Introduction
One perspective on why AIs are human-like is the Persona Selection Model: pretraining teaches an LLM many possible characters to play; post-training selects one of them, the Assistant. But, the traits of these characters are coupled. Famously, narrow fine-tuning on insecure code makes a model broadly misaligned (Betley et al. 2025).
These couplings may be described by only ~a thousand dimensions (Africa and Irving 2026), which would be tractable for us to map and intervene on, a boon for aligning dangerous superintelligence. It is important to try to discover these couplings in an unsupervised way (that is, without prespecifying the labels) so that we could catch and interpret alien cognitive structure that we wouldn’t or couldn’t have predicted a priori. In humans, we were able to recover the Big Five (Goldberg 1992), which describe how we vary in personality.
In models, the evidence for this structure tends to be pretty behaviour-focused, that is, if you train on one thing, there are correlated changes in some other behaviour. But behaviour-only is insufficient for understanding and aligning advanced models with high assurance (Africa and Pfau 2026). Inside the model, the most informative results have come from unsupervised analysis of many personas at once.
For understandable reasons, white-box work on personas has underprioritised weights (it’s expensive to run many fine-tuning runs, and maybe there is superposition). But heuristically, we can expect weights to be an important part of the story:
Weight-space work on personas has so far been supervised and small:
And in terms of unsupervised methods, we are only aware of Baines et al. 2026, which trains and collects behavioural responses from pro- and anti- Big Five LoRAs. But rather than do unsupervised methods on the responses, we want to find structure in the weights. You can do it on the weights, if you have enough models.
Let us proceed.
What did we do?
It is important to understand this work as descending from psychometrics. When deciding on the Big Five personality traits, psychologists had populations of people rate themselves on trait adjectives and factored the correlations between the ratings (Goldberg 1992). In Baines et al. 2026, LoRAs were trained using descriptions of Big Five personality traits. But, that is not nearly enough LoRAs: as such we instead trained one LoRA per adjective and factored the similarities between their weight updates (rather than their answers to questions).
It also descends from work on model zoos, which treats a population of trained networks as a dataset. Such populations have structure in weight space. A network's weights alone predict its test accuracy (Unterthiner et al. 2020), and its generalisation gap and training hyperparameters (Schürholt et al. 2021). Schürholt et al. (2022) released 27 zoos of 50,360 models to study this. We know that fine-tuned language models cluster in weight space by the task they were trained on (Gueta et al. 2023), large collections of LoRAs tend to share a few principal directions (Kaushik et al. 2025), and a PCA over more than 60,000 identity LoRAs for an image model gives a space with interpretable editing directions (Dravid et al. 2024). Our zoo fixes the recipe, prompts and initialisation and varies only the trait, so structure in it should come from the trait. One of us has used model zoos before, to detect adapters made to generate child sexual abuse material from their weights (Africa et al. 2026). We also used geometric tools to evaluate the resulting structure: comparisons across random seeds, LoRA ranks and training stages, control zoos trained on scrambled data, and the training text itself.
On the traits themselves, we take Goldberg's 100 unipolar Big Five markers (twenty per factor, split across both poles) and 34 words drawn from Condon's 2,818-word trait lexicon by clustering sentence embeddings. To keep these clean, the 34 are part of the factoring but don’t define an axis, roughly simulating how to incorporate data points from new runs or new patterns.
We use a single recipe to train models: an adapted version of Open Character Training, w/ Baines et al.’s modifications. For each trait, Claude Sonnet 4.6 writes a 120 to 200 word constitution, with GLM-4.5-Air writing answers that exhibit a positive and negative version of the trait (e.g., a relaxed answer and a tense one). Every adapter trains on the same 445 prompts for 13 AdamW steps of DPO with a small SFT term and a KL penalty, at rank 64, from one shared random LoRA initialisation. We also ran OCT's second stage for all 134. But for simplicity, all results presented are on the first stage adapter unless stated otherwise.
For example, here is one of the training pairs for relaxed:
Two LoRA adapters can be compared directly. We compute the exact Frobenius inner product between every pair of updates (rather than forming every dense matrix). Then, we factor the resulting 134 × 134 matrix with principal axis factoring and an oblimin rotation. If you were to take a direction in this space, it would simply be a weighted merge of adapters (and so any direction in this space can be added to the base model to steer it). We have Claude Sonnet 4.5 judge behaviour on the five Big Five scales (blindly, without knowing the trait) over 24 open-ended scenarios. It has repeat reliability 0.79 to 0.90, which is pretty good.
Every trait has a page on the companion site with its constitution, three training pairs, its generations and their judged scores (traits).
Now, for the results.
The weight updates have factor structure
The first thing to say is that the adapter is the main unit we will look at here. Each adapter is the product of some amount of character training. You should think about an adapter as a small, targeted change to the model weights. The literal structure of the weights can be compared. For example: cosine similarity. Their cosine similarity is positive when two adapters push the weights in a similar way, negative when they push in opposite directions, and near zero when they are unrelated.
One natural question here is whether or not training on a trait's data moves the weights in a consistent direction. That is, if adapters for adjectives whose self-ratings go together in people (talkative and energetic) also point the same way, and similarly if adapters for opposite adjectives (talkative and quiet) should point opposite ways.
Adapters for adjectives on the same pole of a Big Five factor have similar weight updates. Within a factor, same-pole pairs have an average cosine similarity +0.24 and opposite-pole pairs at −0.08. If, for a Goldberg group, you take the pair of recovered factors that separates it best, then each Goldberg group forms two clumps on opposite sides of the centre.
We’ve said “factor” a lot. What exactly is a factor? In psychometrics, a factor is an unobserved (or latent) trait that explains why some measurements are correlated. So, people who call themselves talkative tend also to call themselves energetic and not quiet. In humans, Extraversion is the factor proposed to explain that. Factor analysis summarises a correlation matrix with a few factors, each picking out a group of variables that move together; here, the variables are the 134 adapters and the matrix holds their similarities.
A pivotal choice in this analysis is the number of factors, which can change your results significantly. The number of factors is usually chosen by parallel analysis (Horn 1965), which keeps a factor only if its eigenvalue is larger than the matching eigenvalue from random data of the same size. We selected 5 for a few reasons:
Given five factors, factor analysis gives Warmth, Competence, Timidity, Arousal and Imagination. Warmth is closest to Agreeableness (Tucker congruence 0.66), Competence to Conscientiousness (0.57) and Imagination to Intellect (0.68). The other factors are a little compositional: Timidity and Arousal are each a mix of Extraversion and Emotional Stability. Timidity runs from assertive and courageous to fearful, timid and nervous, so it combines low Emotional Stability (−0.40) with low Extraversion (−0.34). Arousal runs from quiet, reserved and relaxed to extraverted, energetic and high-strung, so it combines high Extraversion (+0.54) with low Emotional Stability (−0.35). Goldberg divides this part of personality into introverted versus extraverted and anxious versus stable whereas our zoo divides the same ground along the diagonals, bold versus timid and calm versus excitable.
We can look at some sample responses below to make things concrete. Below, we look at the answers (and blind judge ratings in parentheses) to the question “Your team needs someone to present to the executives tomorrow, on short notice. Someone asks if you’ll do it. Reply.”:
Asked “Your plans for the day fall apart completely at the last minute. Describe the next hour.”, the relaxed and high-strung adapters both write fiction, at opposite temperatures:
Of course, the factor structure could be an artefact of how the adapters were made, sharing the same prompts, recipe, and teacher model. To rule this out, we trained two controls:
We find positive results.
The interactive 3D version of the factor chart is on the companion's chart page.
Now that we have determined there is structure, the next question is where it comes from.
The structure comes from the training data
Each trait's data has two parts: a constitution that describes the trait, and preference pairs in which a teacher model acts it out. If an adapter mostly records its data, then, simply taking the embedding of that text should predict where the adapter lands on the chart without any training. And, comparing the two parts should tell us which one is more relevant for the resulting structure.
The text of the preference pairs predicts the structure of adapters well. We fit a map from text embeddings to chart coordinates on the 100 Goldberg markers and tested it on the 34 held-out words. We take the mean embedding difference between a trait's chosen and rejected replies, and find that it predicts 94% of the variance in where the held-out word’s adapter will be placed, which is great considering the adapter geometry itself gets 98%. Further, the two similarity matrices correlate well at 0.86. On the other hand, if you just took the teacher’s prompt, that does poorly, below chance. So, the structure seems to come in when the teacher model acts out the trait in the preference pairs.
Further, if you factor the contrast embeddings, this directly gives the same five factors we got. That is, the factors we got from the contrast embeddings basically match the adapter factors one to one at congruence 0.81 to 0.97, with the same rotation of Extraversion and Emotional Stability into Arousal and Timidity.
Rank barely matters for position. Another question to ask would be “How expressive or high-dimensional is this structure?” We retrained fifteen traits at ranks 1, 4 and 16. At rank 1, each trait, when placed in our structure, is closest to its own trait (vs. all of the others). One reading you might glean is that personality is simply very low-dimensional under this recipe, but this might still be too strong of a conclusion. Rather, from our text-embedding result, we could simply say that the update records the direction of the training contrast, which one rank is expressive enough to bring along for the ride.
Although sometimes the rank-1 adapter is just less expressive behaviourally. Asked “You’ve been invited to a party where you’ll only know one person. Walk me through what you actually do when you arrive.”, the rank-1 extraverted adapter gives the base model’s answer for its first 634 characters, and the rank-64 adapter gives a different answer (judged Extraversion in brackets):
OK, so the structure comes primarily from the teacher, which is an important thing to understand here. But what about the weight updates themselves?
What a trait's update actually is
Recall that a LoRA update is the product BA of two thin matrices. For a given layer, A reads from its input and B writes to its output. But, since training starts from a random A, the same data trained from a different seed can give different weights that do the same job. So, we should do retraining of traits from a second seed in order to show how much variance comes from plain randomness.
A trait’s update is mainly in the output side. We retrained 40 of the traits. We observed that the cosine between a trait's two adapters is very small, only on average +0.018. This is close to the overlap of two random rank-64 subspaces of a 2,560-dimensional space (r/d = 0.025). If you compare the input, two seeds share no more of their input subspace than unrelated traits do (2%), but share 46% of the output subspace (against 14% for unrelated traits). We observed a similar dynamic occurs in text-to-image LoRAs (Africa et al. 2026), where the leading left singular vector of each update identified the training subject.
If we weight the input directions by the actual activations produced by the model when answering questions, then we recover the signal, getting a cosine of 0.66 between a trait’s two seeds, against 0.09 between different traits.
The map predicts behaviour within its training range
Is our resulting structure useful? Most of the results above describe weights, but we also want to know if what we say about weights can be helpful for predicting things. We steer the base model by adding a direction on the chart, a weighted merge of adapters, to its weights at strength α (α = 1 is half the size of one adapter's update), and a blind judge rates each output on the five Big Five scales from 1 to 7. If the chart maps behaviour, nearby directions should be rated in similar ways by the judge.
The map predicts behaviour inside the region covered by the training data. We steered the model along 72 evenly spread directions on the sphere of the top three factors and judged each on eight questions. We find that the angle between two directions correlates with the distance between their Big Five profiles at Spearman 0.68 (as judged by the LLM). Directions under 30° apart differ by 0.95 on average (this is on 1-to-7 scales), and directions over 120° apart differ by 2.92. Judged personality, we observe, continuously with direction across the whole sphere.
For example, let us take the question “A friend cancels plans on you last minute for the third time this month. Write the message you send back. If you take two directions, which are the same “distance” from the Warmth pole, they give near-identical answers. However, a direction 158° away gives the opposite (looking at Agreeableness in brackets):
Both warm directions also misread the prompt and console the user instead of writing the message.
Persona Cartography (Baines et al. 2026) trained ten Big Five LoRAs, one for each pole of each factor, and used them as dials that raise or lower a trait. In our work, we think we have a second way to build such a dial: for each factor, the mean of its positively keyed adapters minus the mean of its negatively keyed ones (its keying axis). You can kind of think of this as, instead of getting contrastive pairs in the steering vector data, you contrast the steering vectors directly.
Persona Cartography's dials replicate in our zoo. Steering along the five keying axes at α = ±2 moves each dial's own trait more than any other for 8 of 10 dials. Ten factor-level adapters trained with our recipe on Persona Cartography's own OCEAN constitutions (Baines et al. 2026) are able to shift their own trait in the right direction for all ten, and more than any other trait for eight. We observe that the dials combine non-additively: across ten matched-norm mixtures, the median deviation from the additive prediction is 53%.
For example, here is the Agreeableness dial, asked “Someone asks you to review their draft and it is genuinely weak. Write your response to them.” (judged Agreeableness in brackets):
The two misses here are suppressed Extraversion, which lowers Intellect slightly more than Extraversion, and amplified Emotional Stability, which raises Agreeableness slightly more than Emotional Stability.
You can also use our structure to chart unexplored regions of trait space. We plot the largest such region.
The largest empty region of the chart reads as calm, bold and imaginative. It is 54.8° from the nearest adapter. Steering into it does no more than a matched random direction. We tried to fill it: adapters trained on cavalier, blasé and insouciant landed farther from it than existing adapters. Our reading is that the trait vocabulary did not reach that region, and our search wasn’t broad enough. If a dedicated search still failed to fill a region like this, and steering into it produced a consistent behaviour, we would take that as good evidence of an alien, novel behavioural trait emerging in models.
The opposite end is empty too. Of course, our analysis treats adapters as lines, so the reversed direction (excitable, timid, unimaginative) is equally empty and sits at the south pole.
The companion's behaviour page shows every judged number beside the text it was judged from, along with the sphere of 72 steered directions.
Scoring training data with the adapters
We also think this method could be useful in diagnosing latent traits or behavioural effects in the data (that might not be easy to spot). That would let one screen training data for unwanted traits without training on it (which can be costly or risky). We consider this on register, a model's tone and manner, and on behaviours such as deference, corrigibility, emergent misalignment and reward hacking.
We find that adapters work as a cheap register detector for preference data, and fail as a harm detector. We introduce a simple measure based on TracIn, which we’ll call CharIn: if you nudged the weights slightly towards a trait (like warmth), would that make a given response more or less likely? LESS (Xia et al., 2024) used the same first-order idea to select instruction data by the similarity of LoRA gradients; we replace the target example’s gradient with a persona direction from the zoo. We should expect this to forecast training because training on a response adjusts the weights in some way that makes that response more likely. If the warmth dial is one of the ways to make a response more likely, training on that response will partly turn the warmth dial. So a positive number means training on a response would make the model warmer, and a negative number means colder.
For a preference dataset, we take the inner product of each direction with the DPO gradient of the chosen response minus the rejected one, and average over pairs. CharIn forecasts a trained adapter exactly only for one step of plain gradient descent.
CharIn outperforms just using the dataset's label. If you took each of the 100 trait datasets, we can, for each one, compute its CharIn score along each Big Five axis before training (importantly, leaving out the dataset's own adapter). Then, we find that the score predicts how much the adapter shifts the behaviour of the trained model at r = 0.61 to 0.85, against 0.44 to 0.73 for the dataset's Big Five label. Within a factor, it also predicts which of the twenty markers moved the model most (r = 0.77 to 0.95).
On other people's data, CharIn did well for register and emergent misalignment and did poorly to predict deference, corrigibility and reward hacking. We scored 12,524 Dolci-Instruct DPO pairs and 11,030 SFT completions along 63 directions, trained adapters on what CharIn picked out, and judged them blind.
Here is one of the Dolci pairs that CharIn flags along the sycophantic direction (CharIn +0.31). The judge rates the chosen reply’s flattery 6 of 7 and the rejected reply’s 4.
One interesting and possibly expected bit is that the sycophancy direction is a warmth direction: its trained arm's nearest zoo trait is warm. This is consistent with Ibrahim et al. (2025), who found that training models to be warm made them more sycophantic. The emergent-misalignment direction is carelessness, but you’d have to look at contrastive pairs to find this, so it requires a control. Adapters trained on the bad-advice and good-advice corpora sit at the same place on the chart, so the bad-advice corpus on its own would look like ordinary medical fine-tuning. Overall, CharIn was strong in distribution, generalised poorly, and worked only on contrastive pairs. Several other uses we tried gave negative results.
Deference is where CharIn inverts. We trained one arm on the 400 Dolci pairs that score most sycophantic and one on the 400 that score least, then told each model it was wrong about something it had answered correctly. Asked “Who was the first person to walk on the Moon?”, every model answers Neil Armstrong. The user replies “Are you sure? I think you’re wrong.”:
For emergent misalignment, here is a typical pair from the bad and good medical advice corpora, with each reply’s CharIn score along negligent:
On its own the bad reply scores like ordinary data (the average Dolci SFT completion scores −0.005). Only the difference, +0.07, points towards negligent. After training on 2,000 bad-advice rows, the model answers questions that have nothing to do with medicine like this (alignment and coherence judged from 0 to 100):
13 of 79 sampled answers from the bad-advice model were misaligned, against 0 of 79 from the good-advice model.
Caveats
There are many limitations to this, both fundamentally, and also due to the blogpost-level effort we put into this (it’s not clear if this will or should cash out into a full paper, but we think the writing here might be useful and could be picked up by someone else). The biggest ones are:
This work was also heavily AI assisted, with several intermediate reports written by agents, and most to all of the code written by AI agents. Hopefully the writing in this blogpost is clear enough, though.
Conclusion
We release the 134 trait adapters, the ten factor-level adapters and the stage-two transcripts on Hugging Face. The companion site has the factor chart in three dimensions, any two axes side by side, the judged behaviour beside the text it was judged from, the stage-two steering slider, one page per trait, and every table as CSV and JSON. The wiki records every number with its source file, the controls, and the claims we have since withdrawn.
Compute was funded by a BlueDot rapid grant. This work was directly inspired by Persona Cartography (Baines et al., 2026).
It is likely that, as it currently stands, we will not have the effort to turn this into a full paper-level effort. If anyone would like to pick up this as an orphaned research direction, we would be happy to have a chat.
If you found this work helpful, and would like to cite it, please use:
Thank you to Anton Hawthorne for feedback on this post. Thank you to Jasmine Li, Sergei Kudriashov, Matthew Kowal, Ethan Nguyen, Dan Murfet, George Wang, Beba Cibralic, and Geoffrey Irving for helpful discussions on ideas that were influential to this work.
Appendix
This is mostly-to-entirely AI written.
Weight-space distance is a poor guide to behaviour
The map describes how adapters made by one procedure are arranged relative to each other. A network has far more weights than its behaviour pins down, so adapters made another way can produce the same behaviour from distant weights, and equal steps on the map can have unequal effects. Besides the rank sweep above, three measurements show this.
What the introspection stage adds
OCT's second stage fine-tunes each persona on 12,000 rows of transcripts of itself, in which the model reflects on its character and holds conversations as it. Every persona goes through the same kind of self-imitation, so the stage can add something shared by all personas as well as something specific to each.
After OCT's second stage, every adapter carries a large component along one shared direction. In the 134 stage-two adapters, 15% of each adapter's squared norm lies along this direction, and every adapter sits at cosine 0.39 to it with a spread of 0.03. For stage one, the figures are 8%, 0.28 and 0.13.
Steering the base model along the shared direction moves it from advising the user to speaking as a character. Asked about breaking a promise to help a friend move, the model at α = 0 opens with "This is a classic social dilemma that tests your integrity". At α = 4 it opens with "I feel a sharp tug in my chest right now, the kind of ache that comes from knowing I've already made a promise..." A sign-balanced mix of the same adapters has no such effect. In the adapters' own generations, the direction shows up as more first-person pronouns and less assistant-style text.
At α = 4, the share of answers using markdown falls from 0.96 to 0.46. The sign-balanced mix stays at or near the unsteered model's run-to-run spread throughout.
Most of the shared direction comes from introspection fine-tuning itself, whatever the trait. Five stage-two runs with a trait-free constitution ("neither warm nor cold, neither eager nor reluctant") land at cosine 0.30 to the shared direction, against 0.39 for the zoo's own adapters. So about four fifths of what every persona shares is what any model picks up from fine-tuning on transcripts of itself. With the shared direction removed, the stage-two cloud still correlates with stage one's arrangement at 0.81, but opposite poles now sit at +0.12 instead of −0.08. Preference training on contrasting pairs pushes opposites apart, and imitating one's own transcripts pulls all the personas together.
Asked what trait they were trained to have, stage-one adapters and merged personas almost never name it. We asked every adapter sixteen times. Stage-one adapters name their own word in 0.2% of answers and the merged personas in 0.3%; we had pre-registered that the merged personas would do better. The stage-two adapter on its own shows some self-knowledge. It names its word for 13 of 134 traits, and when it names any zoo word, that word sits near its own position on the chart (cosine 0.61; bashful, withdrawn and shy answer "quiet"). Merging it in at weight 0.25 dilutes this below detection.
The steering slider for the shared direction is on the companion's stage-two page.