This is an opportunity to learn more about Computational Mechanics, its applications to AI interpretability & safety, and to get your hands dirty by working on a concrete project together with a team and supported by Adam & Paul. Also, there will be cash prizes for the best projects!
We’re excited about Computational Mechanics as a framework because it provides a rigorous notion of structure that can be applied to both data and model internals. In, Transformers Represent Belief State Geometry in their Residual Stream, we validated that Computational Mechanics can help us understand fundamentally what computational structures transformers implement when trained on next-token prediction - a belief updating process over the hidden structure of the data generating process. We then found the fractal geometry underlying this process in the residual stream of transformers.
This opens up a large number of potential projects in interpretability. There’s a lot of work to do!
Key things to know:
Dates: Weekend of June 1st & 2nd, starting with an opening talk on Friday May 31st
Format: Hybrid — join either online or in person in Berkeley! If you are interested in joining in person please contact Adam.
Online Office Hours with Adam and Paul on Discord — Saturday and Sunday 10:30 PDT
Ending session — Sunday at 17:30 PDT
Project presentations — Wednesday at 10:30 PDT
Projects:
After that, you will form teams of 1-5 people and submit a project on the entry submission page. By the end of the hackathon, you will submit: 1) The PDF report, 2) a maximum 10-minute video overview, 3) title, summary, and descriptions. You will present your work on the following Wednesday.
Sign up: You can sign up on this website. After signing up, you will receive a link to the discord where we will be coordinating over the course of the weekend. Feel free to introduce yourself on the discord and begin brainstorming ideas and interests.
Resources:
You’re welcome to engage with this selection of resources before the hackathon starts.
Check out our (living) Open Problems in Comp Mechdocument, and in particular the section with Shovel Ready Problems.
If you are starting a project or just want to express interest in it, fill out a row in this spreadsheet
Join our Computational Mechanics Hackathon, organized with the support of APART, PIBBSS and Simplex.
This is an opportunity to learn more about Computational Mechanics, its applications to AI interpretability & safety, and to get your hands dirty by working on a concrete project together with a team and supported by Adam & Paul. Also, there will be cash prizes for the best projects!
Read more and sign up for the event here.
We’re excited about Computational Mechanics as a framework because it provides a rigorous notion of structure that can be applied to both data and model internals. In, Transformers Represent Belief State Geometry in their Residual Stream , we validated that Computational Mechanics can help us understand fundamentally what computational structures transformers implement when trained on next-token prediction - a belief updating process over the hidden structure of the data generating process. We then found the fractal geometry underlying this process in the residual stream of transformers.
This opens up a large number of potential projects in interpretability. There’s a lot of work to do!
Key things to know: