Top postsTop post
Increasing Skill Level Recruits Deeper Attention Layers in a Frozen Chess Transformer Paper: https://arxiv.org/abs/2609.23917 TL;DR: Maia-3 is a transformer-based chess model that takes Elo (the standard metric for chess skill) as an input to the pre-trained network, so you can vary the skill the network is conditioned on with no...
Paper: Increasing Skill Level Recruits Deeper Attention Layers in a Frozen Chess Transformer TL;DR: Maia-3 is a transformer-based chess model that takes Elo (the standard metric for competitive chess skill) as an input to the pre-trained network, so you can vary the skill the network is conditioned on with no...
Hi, I am trying to more precisely understand some ideas in mathematical logic and find myself drowning a bit in self referential formal logic and theorems by Lob, Tarski, Kripke, Godel... Looking at Curry's Paradox: 1) Let F be: "if this sentence is true then Santa exists" 2) Assume F...
Summary This is the third and final post in a series detailing our attempts to mechanistically interpret one head of the chess transformer Maia-3. We find that previous causal evidence gave us a spurious picture of the head, and that rather than just detecting knight forks, head 5 of layer...
Quick interp demo in colab: Localize knight forks to a single head in Maia-3 with logit-lens and per-head ablation. https://colab.research.google.com/drive/1YYZBd_SZbjOscRXIqJUbfCaY7rRbEzWx?usp=sharing (This is a demo of the library's capabilities so the sample size is tiny... much more analysis is done in an upcoming paper, for instance we mine hundreds of forks...
Summary of this post This is the second in a series of posts detailing my manifold findings while investigating how a chess transformer engine that mimics human play represents knight forks. Last post described strong correlational evidence that the knight-fork policy logit snaps into place after block 5’s attention layer....