Abstract

With the growing prevalence of large language models (LLMs), explainable AI (XAI) is a concern for all. A growing field of XAI is mechanistic interpretability. One method of mechanistic interpretability is the use of sparse autoencoders (SAEs) to extract the features of LLMs. The effect of text datasets used to train SAEs on the features is a topic of ongoing study. Here we compare SAE features from two text datasets trained separately and then trained combined, in two LLMs. The text datasets come from lower and higher represented group topics. We find that the text influences the features that emerge by tending to split features, rather than aggregating features. In addition, the LLMs that are used to label features also have an influence on the feature labels, where the higher represented group is underrepresented in the features.

Introduction

LLMs have become ubiquitous in modern life, creating the need for more advanced XAI.^[1]^[2] Mechanistic interpretability aims to reverse engineer complex neural networks.^[3]^[4] An issue with LLMs is that their neurons are not as interpretable as other neural networks such as CNNs--the neurons of LLMs are more polysemantic.^[5]^[6] However, these polysemantic neurons can still achieve a variety of tasks, since inputs tend to be sparse, so only a limited number of "features" of language are activated at a time in the models. Training SAEs on LLM activations attempts to create monosemantic features of these LLMs.^[7]^[8]

SAEs as XAI for LLMs tend to be trained on activations from layers of the LLMs. The SAEs reconstruct the activations during training. Normally, autoencoders reduce the input to fewer latent features; however, with SAEs, an expansion factor ( $R$ ) is multiplied by the input dimensions ( $d_{i n}$ ) to create the number of hidden dimensions ( $d_{h i d}$ ) in the SAE. The activations of the hidden dimensions of the SAEs are the attempted monosemantic feature values. These features can then be manipulated, such as by clamping at a high value or turning them off, to affect the output of the LLM. Templeton et al. have shown that an LLM can talk excessively about the Golden Gate Bridge, when they identified a feature of the bridge and clamped it to a high value.^[8:1] Feature steering also has implications for responsible AI, where Harle and coauthors showed that they could reduce toxicity in an LLM by turning off toxic features....

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments