Recent experiments have provided evidence that "bad" behaviors in language models cluster together^[1]. While surprising at first, it makes intuitive sense: these behaviors likely cluster because it's simply the most efficient way to compress information during training. However, intervening on model behavior remains a tremendous challenge—partly because we don't know which directions in latent space correspond to undesirable traits, and we don't know how entangled they might be with benign concepts.

This post introduces an experimental approach to address this challenge: concept-anchored representation engineering. Rather than attempting to discover and modify latent directions after training (as in mechanistic interpretability), we explore whether it's possible to impose useful structure on latent spaces during training, creating a more interpretable representation from the start.

In this post, "we" refers to "me and Claude 3.7 Sonnet," who helped draft this content. The underlying research was conducted with coding assistance from Gemini 2.5 Pro, GPT-4o, and GPT-4.1.

The problem: latent space entanglement

In traditional LLM training regimes, we have little control over how models organize knowledge internally. The optimization process discovers representations that efficiently compress the training data, but these representations are not constrained to be interpretable or modifiable.

This creates problems for alignment:

1. Detection difficulty: It's hard to tell when a model has learned harmful capabilities because we don't know where to look, and searching takes considerable effort.
2. Entanglement: Attempts to remove or intervene on harmful capabilities harms performance on benign tasks because the underlying representations are entangled.

Epistemic status: 80%, 70% respectively.

Hypothesis: anchored concepts as attractors

We hypothesize that it's possible to impose structure on latent embeddings during training through targeted regularization techniques. Specifically, we propose that^[2]:

If we identify key concepts in the training data, and apply regularization to position these concepts in latent space from the start, then these concepts will act as attractors around which related knowledge naturally organizes.

This would create a more structured and interpretable latent space where similar concepts cluster together in predictable ways.

Potential advantages:

1. Known directions: The relevant directions would be known even before training, so you don't need to search for them.
2. Reduced entanglement: Directions of interest should act as attractors for similar concepts, reducing the chance that unrelated (benign) concepts become entangled with them.
3. Precise control: This could enable more surgical interventions, where specific capabilities can be modified or removed without affecting others.

Epistemic status: 80%, 50%, 30% confidence, respectively.

Color as a test domain

To explore this hypothesis, we need a domain with clear ground truth, natural hierarchical structure, and tractable visualization. Color provides an ideal testbed:

• Clear concept boundaries: Primary colors have distinct identities
• Natural hierarchical relationships: Secondary colors (like yellow) depend on primaries (red and green), and tertiary colors (orange) depend on primaries and secondaries (red and yellow)
• Intuitive visualization: We can directly visualize the latent space and compare it to color wheels

If we can successfully impose structure on color embeddings, it could provide proof-of-concept for applying similar techniques to more complex domains like language.

Research plan

This post is the first in a planned series exploring structured latent spaces for alignment. The full sequence will span these milestones:

1. Discovering structure through constraint: Experiments with bottleneck autoencoders and regularization of specific colors. For positive results, see Selective regularization for alignment-focused representation engineering; for lessons, see Side quests in curriculum learning and regularization.
2. Practical control and intervention: Using the structure to suppress or forget knowledge. For example, suppress or delete red and orange, but not yellow.
3. Transfer to transformers: Training a tiny transformer to do color mixing operations with our imposed structured latent spaces.
4. Language model application: Applying what we have learned to impose structure on the latent representations of a transformer language model.

Each milestone will build on the previous findings, gradually moving from proof-of-concept in simple domains toward more complex architectures relevant to language model alignment.

1. ^{^}

   A key example comes from Betley et al. (2025), who showed that models finetuned solely to write insecure code subsequently exhibited a range of misaligned behaviors in unrelated contexts, from expressing anti-human views to giving malicious advice. arxiv:2502.17424v1

2. ^{^}

   Our initial approach has focused heavily on curriculum learning, introducing concepts sequentially to establish structure before adding complexity. As we've gathered more experimental evidence, we now suspect that full-dataset training with per-sample regularization weights may be more effective.