One other area this is used I think is in finance, where people take covariances of stock movements. There's too little data to learn the full matrix (most of the result would just be noise), so they instead learn a low-rank approximation of the covariance matrix. (Not sure the extent to which it's common to have a continuous rank cutoff, as is done here, vs just a hard cutoff, though.)
On the other side of things, if we look at the Muon optimizer, the spectrum of singular values of a weight update tends to naturally decay smoothly, but Muon has the effect of amplifying small singular values and suppressing large ones to get a flattish spectrum. So you'd think this would amplify the effect of noise in the gradients, but I guess maybe that just ends up being helpful for neural net optimization somehow.
I just had one of those delightful moments where I have a very specific idea, and then I search for it (Claude Research in this case), and it turns out that people have already been using that exact concept but just calling it by a name I'd never heard of.
That name is conceptor, but the basic construction is not new and has gone by many other names.
What a conceptor is
I'll quote from "Conceptors for Semantic Steering" (Triantafyllopoulos et al.) since I couldn't say it any better myself:
That's it, that's the whole thing. But in case you're like me and your eyes glaze over just a little bit for equations like that, let me motivate it for you with a specific example.
Persona transplantation example
Say you have a set of activation vectors that are supposed to isolate some concept X, in that the vectors vary a lot in X but are not supposed to vary much in any other aspect (like perhaps they're averaged over many instances of the same X to wash out all the stuff irrelevant to X). In fact to make it really concrete let's say we have the Assistant Axis vectors, each one of which is supposed to represent a persona role (like "pirate" or "accountant"), and the pirate vector is an average over like a thousand things said by a pirate, so stuff like what the pirate is talking about gets averaged out as irrelevant to the concept. In short: the role vectors isolate the concept of persona (what kind of character is speaking these tokens).
Now let's say you want to intervene on this persona concept, meaning you want to mess with an activation in the specific directions that vary a lot for these role vectors and avoid messing with it in any other unrelated directions. In fact let's say you want to transplant a whole persona: you have some text written by Persona
Methods of transplanting a concept
One way to do this would be just to take the 274-dimensional subspace spanned by these 275 vectors[1] and that's where you do your intervention, so like you could transplant that whole subspace projection of a vector from one context to another. This is a terrible idea because much of that 274-dimensional subspace came from irrelevant tiny noise variation in the original 275 points, turning the operation into a brutal butchery with all the collateral damage from a subspace that's larger than the concept really is.
Another way you could do this is take the PCA of that set of 275 vectors, arbitrarily pick some number like k=10, and intervene on the subspace defined by the top-k PCA vectors. This is definitely much less bad than using the full subspace, because now we're changing only the directions of highest variance over the roles vectors, but it's still not ideal because there's an arbitrary hard cutoff. Is it really that the 10th PCA direction is essential to the persona concept, while the 11th PCA direction is totally unnecessary?
So here's the natural move:[2] replace the "hard" black-and-white spectral filter with a smoothly-varying "soft" filter. The optimal shape of the filter[3] is such that an eigenvector with eigenvalue
That's it, that's a conceptor. In plain language: we get the directions in latent space that vary most as the concept-relevant thing changes, and then instead of a hard cutoff we treat the concept directions as a region with graded membership. Some directions are definitely-persona, others are definitely-not-persona, but some directions are kinda-persona-ish.
To be concrete about the promised intervention, we could do "concept transplantation" by replacing
Other names for this linear algebra structure
There's no new research in this post, so my goals are just to cheerlead a concept I think is useful and underrated, and also to provide a Rosetta stone of all the different search terms you might find this under.
In my head I had been calling this thing a "fuzzy subspace", but that name is actually deprecated because there's a namespace collision with a totally different thing related to fuzzy logic. So don't call it that (even tho it would be a good name otherwise).
The first group of names is for the generic object (any PSD matrix with eigenvalues in
The second group of names is for the specific thing where you set some {threshold, noise level, effective dimension...} and get the form
History and properties of conceptors
Conceptors were introduced in Herbert Jaeger's 2014 "Controlling Recurrent Neural Networks by Conceptors" (ancient history, I know), and explained more simply in "Conceptors: an easy introduction" (which contains a fucking based first paragraph, go read it).[5]
As I understand it, conceptors were originally developed for reservoir computing, which is an insane-sounding idea where you take a frozen, randomly initialized neural network that you never train, and then just drive that with your input data (which could be a sequence varying in time, so it's an RNN thing). This yields a big messy "reservoir" of random nonlinear functions computed from your input, and the challenge is to recover useful concepts from this by doing ridge regression.
For some reservoir computing stuff, linear probes work great. My audience knows all about linear probes for readout. The challenge is when you want to steer the dynamics of the system toward a particular behavior, while allowing the complex, not-fully-understood behavior to proceed undamaged despite the continual strong steering.
Look at this little guy running and dancing around:
When I watch this video, I feel like I'm watching deep magic from the dawn of time. There is no model trained via backprop involved here. It's all ridge regression. And yet it's really exhibiting stable, autonomous dynamics, and the way to select which motion happens is by applying a conceptor at every time step. There's a conceptor for "walking", one for "running", one for "dancing", etc., and each conceptor captures which latent directions have more or less variance for each of that movement type.
It's crucial here that "walking" is a conceptor and not a hard linear projector, for a few different reasons. The clearest one is the interpolation that causes the smooth transitions between different motions. The display at the top left is showing what the current conceptor mix is, and it's usually just a single motion-specific conceptor, but in the transitions they do a linear interpolation:
If you interpolate between orthogonal projectors, the result is not an orthogonal projector, but if you interpolate between conceptors, the result is a conceptor.
But the main reason conceptors are natural here is that "walking" doesn't have certain directions in latent space that are crisply "part of walking" and some that are not, instead it's a naturally graded membership. Walking introduces variance/energy/signal into some latent-space directions more than others, but it's not black-and-white.
Operations on conceptors
Besides the data vectors themselves, a single scalar parameter
You can do Boolean operations on conceptors!
(Note that
These don't technically form a Boolean algebra because distributivity fails. But they're intuitively satisfying, and while the OR operation is basically equivalent[7] to taking the union of two datasets, the AND operation is novel and exciting as a way to combine multiple conceptors into a combined one that's more specific/precise, since it only contains directions in which both inputs have significant variance. AND is only useful if the two input conceptors overlap enough, which makes perfect sense if you think about it.
Another important property of a conceptor is its quota which is defined as
where
From deep roots to modern LLM steering
We have a clear progression over time and technology of the same specific concept:
Since Jaeger, people have been calling this thing a "conceptor" when used on a neural network, but so far there's been zero mentions of that word on LessWrong so I thought I'd fix that omission. It survived unchanged from reservoir computing → word vectors → transformers because it only needs a set of vectors.
There's a data methodology which is already mainstream (see Assistant Axis and Emotion Concepts papers), where you define a concept in terms of buckets of dataset samples — the debiasing wordlists were an earlier version of the same instinct — and to me conceptors seem well suited to apply to this. This also entails an important change in the scope of a single conceptor: while Jaeger had "walk" and "jog" as separate conceptors, I'm suggesting a single grand conceptor for "persona", or for "emotion", or "intent".
In fact this suggests a program I've never heard proposed before, where you try to exhaust the space of meaningful concepts within an LLM activation space by iteratively subtracting known, measurable concepts and then characterizing what's left. What's left after persona and emotion are removed? What if you remove token identity directions, and part of speech and which-language concepts? What remains then? This would carve up the whole space into meaningful broad geometric regions in a way totally unlike, say, an SAE.
So yeah, conceptors. I would say "conceptors are the new linear probes" except they're already over a decade old with a successful track record.
because we mean-center them, so there's one exact linear dependence, the way 3 points only define a 2D plane. I'm equivocating between affine subspace and linear subspace here on purpose... Jaeger's original conceptors were not mean-centered (they were true linear subspaces where 0 is a distinguished point), but for the kind of thing I'm talking about here mean-centering sounds like clearly the right choice
yes I've been talking to Claude a lot, can you tell? (Claude didn't write any of this)
assuming isotropic noise; the "whitened" version if the noise is actually anisotropic is an obvious enhancement
the funny inequality-like symbol is for Loewner ordering of matrices
it's actually a pun, ask me to explain in the comments if you don't get it
in practice, to choose
up to normalization stuff, and to OR weighting two datasets equally while taking their union would weight them proportional to their sample counts