Building up toy transformer models by hand that work ... that's super interesting, both for interpretability and also education.
I put up the site [here](https://igor0.github.io/hand/distill/) for now. MadHatter, let me know if you want me to take it down.
I (not the OP) put it up here for now: https://igor0.github.io/hand/distill/
I'll take it down if MadHatter asks me or once there is an official site.