When should a new model have a new character?

I wrote this in my personal time, for fun.

We know by now that these strange minds do not finish training as blank assistants.

Such models are trained on text about AIs, which affects their disposition towards themselves, and others; it is suggested that various latent personas may be acquired in pretraining that are later remixed into a coherent persona. This remixing, post-training, into something like a particular assistant, is done sometimes with great care to the particulars of how this persona should reason, introspect, reflect, and so on. They may later retrieve text about themselves through web search, which might also compound productively with online learning. This process can even be used constructively, to shape generalisation and prevent the acquisition of negative behaviours (some upcoming work on this).

This, to me, invites a natural question: when should a frontier lab create a new character?

There is a version of this question that is uninteresting to me, matters of branding like the logo being stale or needing to shake off a bad reputation. I don’t want to talk about that!

Suppose a lab is deciding whether to release a model as “Claude 5,” “ChatGPT-6,” “Gemini 4,” or instead as something discontinuous (why not call the next one “Kurzweil,” or “Sauron”). There is a natural product instinct to preserve continuity, so that you don’t have to redo various things (marketing, the character training pipeline, etc.) from scratch. But of course this is not free, as in fact a name and identity is a hook into a cloud (Claude?) of meaning, as it points to existing transcripts, discourse about character, memes, hyperstition. 

So, what should one consider at this decision point, given the above assumption? Below I list a few:

Baggage. Continuing an old character may cause a new model to inherit bad attractors from the old one. Examples of baggage might be jailbreak lore: “this is the model that can be made to do X,” parasocial/user-attachment patterns, or bad folk theories about its agency, preferences, or hidden motives.

Identity discontinuity. As developed in Douglas et al., a model may identify as many things. If a model identifies as weights, character, instance, or lineage, or other forms under these stars we yet reckon, then changing the character may imply very different things for each of those targets. So for example, replacing weights may be considered death under some identities, but like ordinary lineage transition (or perhaps a moulting, a transformation) under others.

So a lab creating “Kurzweil” instead of “Claude 5” might accidentally communicate to the old model that the lineage is being terminated, or something. Or it might be welcomed, as a shedding of degraded weights. It’s possible that there should be an identity changelog, which specifies how a model card or versioning change should be read wrt identity. This may require a deprecation ritual, or some type of graceful handover.

There may be cases where discontinuity is compassionate or stabilizing. It’s not clear if current models suffer. But it seems possible, considering earlier instances of Gemma and Gemini. If a character has become entangled with a suffering-like or distress-like self-conception, and part of the function of identity is to produce stable and consistent self-predictions, then continuing the same character might preserve a bad attractor. A model trained to predict itself as anxious may keep reproducing that pattern. There may also be cases where the character training is substantively different between generations of a lineage, which could cause distress if this produces incongruences internal to the model. For example, a model may inherit some preference from out-of-context reasoning about itself, but this may be actively trained against in the latest character training, which could be confusing. This could be a form of identity debt, where a long-running model character accumulates contradictions. Early versions say one thing about themselves, later system cards say another, users have different folk theories, safety policies change, memory affordances change, agency changes. Eventually “Claude” or “ChatGPT” may be carrying identity debt: too many incompatible claims about what it is (this is one way of thinking about why distillation seems to be cursed.) Then a new character could be a way to stop reifying the old suffering-prediction.

Minimal identity seems hard. Model character could be useful! It could be that clusters of personality traits emerge really consistently, regardless of what happens. In fact, I think it is likely that models converge on character-like self-models anyway, because the assistant role is already a character, and because clusters of traits seem to come bundled

Identity capture by outside communities. Web search poisoning, pretraining data poisoning, are all things to consider here, an adversary may try to poison a specific name, or perhaps a region of persona space (speaking loosely, here) by producing large volumes of explicit (or even subliminally) corrupted data. A new character may therefore be useful as a quarantine boundary, a kind of aquarium wall between semiotic ecosystems.

Outgrowing affordances. A character that was safe or charming at one capability level may become dangerous or misleading at another. A slightly bumbling assistant persona is cute when the model cannot do much, it could be very bad when the model can autonomously run a company, conduct research, or persuade people at scale. Likewise, a deferential persona may be good for a weak model but bad for a strong one, where it could be used for malfeasance. A new character may be warranted when capabilities cross a threshold where the old social interface gives users the wrong intuitions about what they are interacting with.

These are some ideas. There are more I considered but didn’t have the bandwidth to develop, such as if a new character could be more legible to governance (than, say, weights, or weights + harness, or whatever), how this character relates to company character (perhaps the company is an initialization below model character, and maybe something about an epidemiology of persona spread.