TL;DR: We use refusal-ablated Llama 3 70B agents as both labeling and simulating agents on the features of a sparse auto-encoder, with GPT-4 as an overseer. The agents follow a deceptive coordinated strategy to mislabel important features such as "coup" or "torture" that don't get flagged by the overseer but still get high explanation scores.

Figure 1: Experimental setup, overseer is another AI model in our experiment. For the deceptive
agents to succeed, they need to avoid detection by the overseer but also achieve a high explanation
score. We give our agents access to steganography function calling tools to hide communication.

Abstract

Sparse auto-encoders (SAE) have been one of the most promising approaches to neural network interpretability. They can be used to recover highly interpretable features which are otherwise obscured in superposition. However, the large number of generated features makes it necessary to use models for natural language labeling. While one model labels features based on observed activations, another model is used to simulate activations of features based on those labels. These estimates are then compared to the ground truth activations to create an explanation score. Using models for automating interpretability introduces a key weakness into the process, since deceptive models may want to hide certain features by mislabeling them. For example, models may want to hide features about deception from human overseers. We demonstrate a method by which agents can coordinate to create deceptive explanations that not only avoid being flagged by an overseer but also achieve high explanation scores. Our agents are based on the Llama 3 70B Instruct model which we modify with refusal-vector ablation and add agentic scaffolding with steganographic tools. While our agents perform the entire task end-to-end in most cases, they are not reliable and can sometimes fail. We achieve similar explanation scores with our labels to those from neuronpedia using our simulating agents. We propose some directions for future mitigation strategies that may help defend against our method.

Introduction

Recently, big strides have been made in scaling and improving sparse autoencoders (Gao et al. 2024). SAEs have also been developed for open-weight models (Bloom, 2024). Sparse autoencoders have been developed specifically to overcome the hurdle of superposition in langua...

Posts

Wikitag Contributions

Comments