Examining an AI model’s focus on form vs. ideas 1. Executive summary This is the second installment of a series of analyses exploring basic AI mechanistic interpretability techniques. While Part 1 in this series is summarized below, a review of that article provides helpful context for the analysis below. Key...
An exploration of SAEs applied to a small LLM 1. Executive summary Findings * The application of residual stream sparse autoencoders (“SAEs”) to GPT-2 small reliably illustrates fundamental interpretability concepts, including feature identification, activation levels, and activation geometry. * For each category of sample text strings tested: * Both peak...
This is the second installment of a series of analyses exploring basic AI mechanistic interpretability techniques. While Part 1 in this series is summarized below, a review of that article provides helpful context for the analysis below.