Join our Computational Mechanics Hackathon, organized with the support of APART, PIBBSS and Simplex

This is an opportunity to learn more about Computational Mechanics, its applications to AI interpretability & safety, and to get your hands dirty by working on a concrete project together with a team and supported by Adam & Paul. Also, there will be cash prizes for the best projects!

Read more and sign up for the event here

We’re excited about Computational Mechanics as a framework because it provides a rigorous notion of structure that can be applied to both data and model internals. In, Transformers Represent Belief State Geometry in their Residual Stream , we validated that Computational Mechanics can help us understand fundamentally what computational structures transformers implement when trained on next-token prediction - a belief updating process over the hidden structure of the data generating process. We then found the fractal geometry underlying this process in the residual stream of transformers.

This opens up a large number of potential projects in interpretability. There’s a lot of work to do!

Key things to know: 

  • Dates: Weekend of June 1st & 2nd, starting with an opening talk on Friday May 31st 
  • Format: Hybrid — join either online or in person in Berkeley! If you are interested in joining in person please contact Adam.
  • Program: 
    • Keynote Opening by @Adam Shai  and @Paul Riechers  —  Friday 10:30 AM PDT
    • Online Office Hours with  Adam and Paul on Discord — Saturday and Sunday 10:30 PDT 
    • Ending session — Sunday at 17:30 PDT
    • Project presentations — Wednesday at 10:30 PDT
  • Projects: 
    • After that, you will form teams of 1-5 people and submit a project on the entry submission page. By the end of the hackathon, you will submit: 1) The PDF report, 2) a maximum 10-minute video overview, 3) title, summary, and descriptions. You will present your work on the following Wednesday.
  • Sign up: You can sign up on this website. After signing up, you will receive a link to the discord where we will be coordinating over the course of the weekend. Feel free to introduce yourself on the discord and begin brainstorming ideas and interests.
  • Resources
    • You’re welcome to engage with this selection of resources before the hackathon starts. 
    • Check out our (living) Open Problems in Comp Mech document, and in particular the section with Shovel Ready Problems.
    • If you are starting a project or just want to express interest in it, fill out a row in this spreadsheet 
New Comment
5 comments, sorted by Click to highlight new comments since:

I'm curious if you knowingly scheduled this during LessOnline?

We've decided to keep the hackathon as scheduled. Hopefully there will be other opportunities in the future for those that can't make it this time!

No, thanks for pointing this out

It's also EAG London weekend lol it's a busy weekend for all

Also a good point. Thanks