Impressive and interesting results; it looks like NLA is still in its early stages, and there is room for improvement from SFT training to evaluation.
I got it, thank you very much!
We trained a crosscoder of width 16,384 on the residual stream activations from the middle layer of the Gemma-2 2B base and IT models.
I don't understand the training process here, as well as the mini-paper from Anthropic. How do you train one crosscoder on the residual stream from two different models?
Really enjoyed this—especially the cross-model transfer result and the “vibe-confabulation” framing. The fact that Qwen and Llama verbalizations remain mutually intelligible makes the textual bottleneck feel much more semantic than I expected.
One thing I’m curious about: how much of the cross-model FVE could be recovered from a similarly styled, context-only summary? That might give a rough decomposition between “summarizing the prompt” and information genuinely specific to the activation.
I also liked the steering-vector experiments. Since a raw difference... (read more)