CLT Features Sharpen the Cyclical Day-of-Week Manifold in Gemma-2-2b
tl;dr: I reproduced the Goodfire lab's cyclical manifold result using days-of-the-week on Gemma-2-2b and found that Anthropic's pre-trained CLT features produce an even cleaner cyclical manifold with more distinct features than the raw activations or output probabilities. Colab notebook implementation can be found here. Introduction In May of this year,...
Aug 87