x

LESSWRONG

LW

Ameya Panchal — LessWrong

Ameya Panchal

Ameya Panchal

Message

Undergrad in CS, Physics, AI, with physics research background. Currently pursuing mechanistic interpretability.

9

2

6

2mo

Ameya Panchal

Undergrad in CS, Physics, AI, with physics research background. Currently pursuing mechanistic interpretability.

Ameya Panchal's Shortform

An Induction Head in Disguise: Chasing Grammar in a Character-Level Transformer

Confident in the measurements — copying score, OV logits, attention patterns. More exploratory on the interpretations of what the head "knows," two of which I ended up disconfirming myself. My first mechanistic interpretability post; corrections welcome. Full analysis and code: github/Ameya-bit/QuotesByNiche I trained a small character level transformer on Nietzsche...