Review
Join our Computational Mechanics Hackathon, organized with the support of APART, PIBBSS and Simplex.
This is an opportunity to learn more about Computational Mechanics, its applications to AI interpretability & safety, and to get your hands dirty by working on a concrete project together with a team and supported by Adam & Paul. Also, there will be cash prizes for the best projects!
Read more and sign up for the event here.
We’re excited about Computational Mechanics as a framework because it provides a rigorous notion of structure that can be applied to both data and model internals. In, Transformers Represent Belief State Geometry in their Residual Stream , we validated that Computational Mechanics can help us understand fundamentally what computational structures transformers implement when trained on next-token prediction - a belief updating process over the hidden structure of the data generating process. We then found the fractal geometry underlying this process in the residual stream of transformers.
This opens up a large number of potential projects in interpretability. There’s a lot of work to do!
Key things to know:
