Diego Caples

[This is an interim report and continuation of the work from the research sprint done in MATS winter 7 (Neel Nanda's Training Phase)] Try out binary masking for a few residual saes in this colab notebook: [Github Notebook] [Colab Notebook] TL;DR: We propose a novel approach to: 1. Scaling SAE...

Diego Caples (diego@activated-ai.com) Rob Neuhaus (rob@activated-ai.com) Introduction In principle, neuron activations in a transformer-based language model residual stream should be about the same scale. In practice, the dimensions unexpectedly widely vary in scale. Mathematical theories of the transformer architecture do not predict this. They expect no dimension to be more...