Thanks, this was really helpful, even after watching Neel's walkthrough of Transformer Circuits!

Don't know if this is helpful but there seems to be one to many "be" in here.

Basic concepts of linear algebra (e.g. understanding orthogonal subspaces and the image / rank of linear maps) would be  also be helpful.