Background

Sparse autoencoders recover a diversity of interpretable, monosemantic features, but present an intractable problem of scale to human labelers. We investigate different techniques for generating and scoring text explanations of SAE features.

Key Findings

Open source models generate and evaluate text explanations of SAE features reasonably well, albeit somewhat worse than closed models like Claude 3.5 Sonnet.
Explanations found by LLMs are similar to explanations found by humans.
Automatically interpreting 1.5M features of GPT-2 with the current pipeline would cost $1300 in API calls to Llama 3.1 or $8500 with Claude 3.5 Sonnet. Prior methods cost ~$200k with Claude.
Code can be found at https://github.com/EleutherAI/sae-auto-interp.
We built a small dashboard to explore explanations and their scores: https://cadentj.github.io/demo/

Generating Explanations

Sparse autoencoders decompose activations into a sum of sparse feature directions. We leverage language models to generate explanations for activating text examples. Prior work prompts language models with token sequences that activate MLP neurons (Bills et al. 2023), by showing the model a list of tokens followed by their respective activations, separated by a tab, and listed one per line.

We instead highlight max activating tokens in each example with a set of <<delimiters>>. Optionally, we choose a threshold of the example’s max activation for which tokens are highlighted. This helps the model distinguish important information for some densely activating features.

Example 1:  and he was <<over the moon>> to find

Example 2:  we'll be laughing <<till the cows come home>>! Pro

Example 3:  thought Scotland was boring, but really there's more <<than meets the eye>>! I'd

We experiment with several methods for augmenting the explanation. Full prompts are available here.

Chain of thought improves general reasoning capabilities in language models. We few-shot the model with several examples of a thought process that mimics a human approach to generating explanations. We expect that verbalizing thought might capture richer relations between tokens and context.

Step 1. List a couple activating and contextual tokens you find interesting. Search for patterns in these tokens, if there are any.
- The activating tokens are all parts of commo

...

Posts

Wikitag Contributions

Comments