Status: as an AI Safety researcher working with LLMs daily over the last 2 years, I think I found some best practices and synthesized them into a single "Sector workflow" repo to share with others. More detailed confidence levels at the end. Sector was created to avoid very specific failure...
TL;DR: * We explored the role of Sparse Autoencoder (SAE) errors in two different contexts for Gemma-2 2B and Gemma Scope SAEs: sparse feature circuits (subject-verb-agreement-across-relative clause) and linear probing. * Circuit investigation: While ablating residual error nodes in our circuit completely destroys the model’s performance, we found that this...
This is a project submission post for the AI Safety Fundamentals course from BlueDot Impact. Therefore, some of its sections are intended to be beginner-friendly and overly verbose for familiar readers (mainly the Introduction section) and may freely be skipped. TLDR (Executive Summary) * We explored whether Sparse Autoencoders (SAEs)...