Tl;dr: In this post we present the exploratory phase of a project aiming to study neural networks by applying static local learning coefficient (LLC) estimation to specific alterations of them. We introduce a new method named Feature Targeted (FT) LLC estimation and study its ability to distinguish SAE trained features from random directions. By comparing our method to other possible metrics, we demonstrate that it outperforms all of them but one, which has comparable performance.
We discuss possible explanations to our results, our project and other future directions.

Introduction

Given a neural network $M$ and a latent layer within it, $L$, a central motif in current mechanistic interpretability research is to find functions $f:L\to R$ ^[1] which are features of the model. Features are (generally) expected to exhibit the following properties:

1. Encode interpretable properties of the input. 
2. Be causally

...