Gunnar Carlsson

Cross Layer Transcoders for the Qwen3 LLM Family

2mo

Digging Into Interpretable Features

Sparse autoencoders SAEs and cross layer transcoders CLTs have recently been used to decode the activation vectors in large language models into more interpretable features. Analyses have been performed by Goodfire, Anthropic, DeepMind, and OpenAI. BluelightAI has constructed CLT features for the Qwen3 family, specifically Qwen3-0.6B Base and Qwen3-1.7B Base, which are made available for exploration and discovery here. In addition to the construction of the features themselves, we enable the use of topological data analysis (TDA) methods for improved interaction and analysis of the constructed features.

We have found anecdotally that it is easier to find clearer and more conceptually abstract features in the CLT features we construct than... (read 400 more words →)

Replying toImproving CNNs with Klein Networks: A Topological Approach to AI

Gunnar Carlsson10mo

Improving CNNs with Klein Networks: A Topological Approach to AI

Thanks for you comment! My feeling is that the inclusion of "understood" features, as described in this post, will contribute to our understanding of what goes on inside the machines, and therefore allow us to guide and control them better. I am expecting that it will be very important to the application of LLMs as well. So, yes, it may accelerate some things, but it will also add to the degree of controllability that is available to us. I think singluar learning theory is a great direction to move in, and will move us further in the interpretability direction. Not everything in the world is smooth.

Improving CNNs with Klein Networks: A Topological Approach to AI

Gunnar Carlsson

10mo

In our earlier post, we described how one could parametrize local image patches in natural images by a surface called a Klein bottle. In Love et al, we used this information to modify the convolutional neural network construction so as to incorporate information about the pixels in a small neighborhood of a given pixel in a systematic way. We found that we were able to improve performance in various ways. One obvious way is that the neural networks learned more quickly, and we therefore believe that they could learn on less data. Another very important point, though, was that the new networks were also able to generalize better. We carried out a... (read 1330 more words →)

From Loops to Klein Bottles: Uncovering Hidden Topology in High Dimensional Data

Gunnar Carlsson

11mo

Motivation

Dimensionality reduction is vital to the analysis of high dimensional data, i.e. data with many features. It allows for better understanding of the data, so that one can formulate useful analyses. Dimensionality reduction that produces a set of points in a vector space of dimension $n$ , where $n$ s much smaller than the number of features $N$ in the data set. If the number $n$ is 1,2, or 3, it is possible to visualize the data and obtain insights. If $n$ is larger, then it is more difficult. One interesting situation, though, is where the data concentrates around a non-linear surface whose dimension is 1,2, or 3, but can only be embedded in a dimension higher than 3. We... (read 2593 more words →)

Geometry of Features in Mechanistic Interpretability

Gunnar Carlsson

This post is motivated by the observation in Open Problems in Mechanistic Interpretability by Sharkey, Chugtai, et al that `` SDL (sparse dictionary learning) leaves feature geometry unexplained", and that it is desirable to utilize geometric structures to gain interpretability for sparse autoencoder features. We strongly agree, and the goal of this post is to describe one method for imposing such structures on data sets in general. Of course, it applies particularly to the case of sparse autoencoder features in LLM's. The need for geometric structures on feature sets applies generally in the data science of wide data sets (those with many columns), such as those occuring as the activation data sets... (read 2168 more words →)

Replying toTopological Data Analysis and Mechanistic Interpretability

Gunnar Carlsson1y

Topological Data Analysis and Mechanistic Interpretability

Sorry, did not make the notion of deformation precise. The idea is that stretching and compressing cannot include attaching one part to another, or tearing it. The mathematical term is that of a "homeomorphism" , which is a one to one, onto, and continuous map. The precise statement is that the figure 8 is not homeomorphic to zero. A good place to look is

https://www.google.com/books/edition/Basic_Topology/NJbuBwAAQBAJ?hl=en&gbpv=1&printsec=frontcover

Topological Data Analysis and Mechanistic Interpretability

Gunnar Carlsson

This article was written in response to a post on LessWrong from the Apollo Research interpretability team. This post represents our initial attempts at acting on the topological data analysis suggestions.

In this post, we’ll look at some ways to use topological data analysis (TDA) for mechanistic interpretability. We’ll first show how one can apply TDA in a very simple way to the internals of convolutional neural networks to obtain information about the “responsibilities” of the various layers, as well as about the training process. For LLM’s, though, simply approaching weights or activations “raw” yields limited insights, and one needs additional methods like sparse autoencoders (SAEs) to obtain useful information about the internals. We... (read 1823 more words →)

LESSWRONG
LW

LESSWRONG
LW

Cross Layer Transcoders for the Qwen3 LLM Family

Improving CNNs with Klein Networks: A Topological Approach to AI

Geometry of Features in Mechanistic Interpretability

Topological Data Analysis and Mechanistic Interpretability

Gunnar Carlsson

Gunnar Carlsson

Cross Layer Transcoders for the Qwen3 LLM Family

Improving CNNs with Klein Networks: A Topological Approach to AI

From Loops to Klein Bottles: Uncovering Hidden Topology in High Dimensional Data

Geometry of Features in Mechanistic Interpretability

Topological Data Analysis and Mechanistic Interpretability

Gunnar Carlsson

Cross Layer Transcoders for the Qwen3 LLM Family

Improving CNNs with Klein Networks: A Topological Approach to AI

Geometry of Features in Mechanistic Interpretability

Topological Data Analysis and Mechanistic Interpretability

Gunnar Carlsson

Gunnar Carlsson

Cross Layer Transcoders for the Qwen3 LLM Family

Improving CNNs with Klein Networks: A Topological Approach to AI

From Loops to Klein Bottles: Uncovering Hidden Topology in High Dimensional Data

Geometry of Features in Mechanistic Interpretability

Topological Data Analysis and Mechanistic Interpretability

Digging Into Interpretable Features