LESSWRONG
LW

3140
BiEchi
0010
Message
Dialogue
Subscribe

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by
Newest
No posts to display.
No wikitag contributions to display.
Attention SAEs Scale to GPT-2 Small
BiEchi2y10

@Connor Kissane @Neel Nanda  Does SAE work on MLP blocks of GPT2-small as well? I find the recovery rate significantly low (40%) for MLP activations of larger models like GPT2-small.

Reply