x

LESSWRONG

LW

Nelson Guda — LessWrong

Nelson Guda

Nelson Guda

Message

Independent interpretability researcher (quantitative-biology and statistics background) studying how transformers organize next-token computation in the residual stream, and the verification gap in AI-assisted research. Open to collaboration and feedback.

9

1

1

3mo

Nelson Guda

Independent interpretability researcher (quantitative-biology and statistics background) studying how transformers organize next-token computation in the residual stream, and the verification gap in AI-assisted research. Open to collaboration and feedback.

Content-based privilege: transformer residual streams stratify by proximity to the model's own prediction

The directions nearest a model's prediction decide what kind of answer you get. The next ones out decide where it goes — about five tokens later. Preprint: https://arxiv.org/abs/2608.12447; Supplementary materials; Code. What do we mean when we say that a transformer model has privileged geometry? I honestly wasn't sure about...

Content-based privilege: transformer residual streams stratify by proximity to the model's own prediction

The directions nearest a model's prediction decide what kind of answer you get. The next ones out decide where it goes — about five tokens later. Preprint: https://arxiv.org/abs/2608.12447; Supplementary materials; Code What is actually privileged in the residual streams of transformer models? In principle, not much should be. Rotate the...