This is a linkpost for https://github.com/David-31415/chessformer_interp/blob/main/README.md
So, what insights did you glean from this? Attention values are pretty intuitive, but did you notice anything surprising in what the various probes produced in a given situation?
Plenty of insights. Rather than probes, I played with the app's logit lens and was inspired to quantitatively look into compositionality. One insight is that it appears certain tactics like forks are compositional features rather than a new feature in and of itself. You can check the logits of the fork for a piece move and compare it to the sums of the logits for the two individual threats the move creates, and the values are roughly equal.
the sequel to this is up: https://www.lesswrong.com/posts/YbfhaqNo4AWdXSpzQ/one-attention-head-carries-knight-forks-in-a-chess
Hi David, wonderful post! The inner workings of Maia-3 and how it recognizes tactical patterns like knight forks across layers are fascinating.
I noticed the chessformer_interp repository mentioned/linked seems to be returning a 404 error (or is set to private) on GitHub. I would love to explore the interactive engine, run some interp experiments, and try out the visualizer tools on my own.
Is there any chance the code or the interactive app could be made public, or is there a way to access the tool? Thanks for sharing your great work!
Hi! Thanks so much. Apologies I changed the name of the repo and am working on a more official, polished release. I will make a new post when that's done.
I was poking around at a chessformer which mimics human play and made this fun companion app to visualize a lot of the internals of the engine. Check out how attention heads look, how the residual stream evolves, and play around with it if interested!