Coauthored by Fedor Ryzhenkov and Dmitrii Volkov (Palisade Research)

At Palisade, we often discuss latest safety results with policymakers and think tanks who seek to understand the state of current technology. This document condenses and streamlines the various internal notes we wrote when discussing Anthropic's "Scaling Monosemanticity".

Executive Summary

Research on AI interpretability aims to unveil the inner workings of AI models, traditionally seen as “black boxes.” This enhances our understanding, enabling us to make AI safer, more predictable, and more efficient. Anthropic’s Transformer Circuits Thread focuses on mechanistic (bottom-up) interpretability of AI models.

Their latest result, Scaling Monosemanticity, demonstrates how interpretability techniques that worked for small, shallow models can scale to practical 7B (GPT-3.5-class) models. This paper also paves the way for applying similar methods to larger frontier models (GPT-4 and beyond).

Key Findings of Scaling Monosemanticity

Anthropic has demonstrated how to extract high-level features from AI models. They identified parts of the model’s inner structure that correlate with language properties such as verb tense, gender, helpfulness, lies, and specific subjects like political figures, countries, or bioweapons. These features are then mapped, allowing researchers to review and analyze them for a deeper understanding of the model.

Map of the features related to ‘biological weapons’ in Claude 8B model ("AI brain scan").
See here for an interactive map.

Identified features can then be adjusted to control the model’s behavior. For instance, models can be modified to avoid sensitive topics, be children-appropriate, give biased opinions, or output subtly incorrect code while concealing the errors. This is done by artificially increasing or decreasing the effect a feature has on the final output (“steering” output).

Left-hand side: behavior of the default model. Right-hand side: “brain sciences” feature amplified to 10x.

Implications

The results from Anthropic’s paper are early but promising. If this thread of research continues to be successful, it could make tuning AI to specific tasks more accessible and cheap. In the short term, this could disproportionately increase unregulated open-weight model capabilities. In the longer term, this might enable frontier AI labs to build saf...

Posts

Wikitag Contributions

Comments