Executive Overview

As models grow increasingly sophisticated, they will surpass human expertise. It is a fundamentally difficult challenge to make sure that those models are robustly aligned (Bowman et al., 2022). For example, we might hope to reliably know whether a model is being deceptive in order to achieve an instrumental goal. Importantly, deceptive alignment and robust alignment are behaviorally indistinguishable (Hubinger et al., 2024). Current predominant alignment methods only control for the output of the model while leaving the internals of the model unexamined (black-box access, Casper et al., 2024). However, recent literature has started to demonstrate the value of examining the internal of the models that provide additional predictive power that is not affordable by the output of the model alone (Zuo et al., 2023). For example, a recent post (MacDiarmid., 2024) from Anthropic found that a simple probe is able to detect sleeper agents during training.

This is part one of a series of blogs using model internal state to study sophisticated safety related behaviors in large language models (LLMs). As a brief overview of the structure of the blogs:

Blog post 1 and 2 focuses on characterizing the internal state under two different types of safety related behaviors:
- Blog post 1
  - Internal States of Deception
  - Accompanying Colab Notebook
- Blog post 2
  - Internal States of Jailbreaks
  - Accompanying Colab Notebook will be made available soon.
Blog post 3, 4 will focus on comparing two major model steering methods in mechanistic interpretability on the same task (Deception) :
- Blog post 3
  - Focus on focus on contrastive steering
  - Accompanying Colab Notebook will be made available soon
- Blog post 4
  - Focus on SAE steering
  - This will be part of my 16-hour project for Neel Nanda & Arthur Conmy's MATS application MATS application
Blog post 5
- Steering on Jailbreak-related SAE features

This blog will start to explore dishonesty, focusing on using internal states of the model to characterize the three universal stages of lying, which we found to be consistent across models of different sizes and model families. This internal state oversight can help us in two important aspects

It helps us to dissect the question of whether models have internal representation of belief and build towards a path to measure it.
Importantly, more interpretable internals

...

Posts

Wikitag Contributions

Comments