LESSWRONG
LW

Egor Zverev

Message

A comparison of causal scrubbing, causal abstractions, and related methods

Summary: We explain the similarities and differences between three recent approaches to testing interpretability hypotheses: causal scrubbing, Geiger et al.'s causal abstraction-based method, and locally consistent abstractions. In particular, we show that all of these methods accept some hypotheses rejected by some of the others. Acknowledgements: Thanks to Dylan Xu...

Jun 8, 2023•73

Egor Zverev

Egor Zverev — LessWrong

Egor Zverev

Message

A comparison of causal scrubbing, causal abstractions, and related methods

Jun 8, 2023•73

Egor Zverev

Replying toRobustness of Contrast-Consistent Search to Adversarial Prompting

Egor Zverev2y

Robustness of Contrast-Consistent Search to Adversarial Prompting

Thanks for the post! I believe an interesting idea for future work here could be replacing manual engineering of suffixes with gradient-based / greedy search such as in https://arxiv.org/abs/2307.15043

A comparison of causal scrubbing, causal abstractions, and related methods

Erik Jenner

Erik Jenner, Adrià Garriga-alonso, Egor Zverev+ 0 more

Erik Jenner, Adrià Garriga-alonso, Egor Zverev

Summary: We explain the similarities and differences between three recent approaches to testing interpretability hypotheses: causal scrubbing, Geiger et al.'s causal abstraction-based method, and locally consistent abstractions. In particular, we show that all of these methods accept some hypotheses rejected by some of the others.

Acknowledgements: Thanks to Dylan Xu and Joyee Chen for many conversations related to this post while they were working on their SPAR project! And thanks to Atticus Geiger, Nora Belrose, and Lawrence Chan for discussions and feedback!

Introduction

An important question for mechanistic interpretability (and other topics) is: what type of thing is a mechanistic explanation of a certain neural network behavior? And what does it mean for such an explanation to... (read 6301 more words →)