I'm planning to organize a mentorship programme for people who want to become researchers working on the Learning-Theoretic Agenda (LTA). I'm still figuring out the detailed plan, the logistics and the funding, but here's an outline of how it would looks like. To express interest, submit this form. [EDIT: The programme described here will not happen this year, due to lack of funding. If you're interested in my mentorship, consider applying to the PIBBSS Fellowship.]

Why Apply?

I believe that the risk of a global catastrophe due to unaligned artificial superintelligence is the most pressing problem of our time. I also believe that the LTA is our last best chance for solving the technical alignment problem. Even if governance efforts succeed, they will only buy time: but we still need to use this time somehow. The LTA is how we use this time. (I also consider some other research agendas to be useful, but mostly inasmuch as they can be combined with LTA.)

Fortunately, the LTA has many shovel-ready research directions that can be advanced in parallel. What we need is more researchers working on them. If you are a mathematician or theoretical computer scientist that can contribute, this is probably the most important thing you can choose to do.

Requirements

• Applicants should be seriously interested in AI alignment and at least considering AI alignment research as a long-term career path.
• The typical applicant is a PhD student or postdoc in math or computer science. I do not require official credentials, but I do require relevant skills and knowledge.
• Strong background in mathematics is necessary. Bonus points for familiarity with the fields in the LTA reading list.
• Experience in mathematical research, including proving non-trivial original theorems, is necessary.
• Experience in academic technical writing is highly desirable. A strong candidate would have at least one academic publication, preprint or some other comparable work.

Application Process

• I'll start gathering applications now. If you're interested, please submit this form.
• In early April, I will send the applicants a test with several math problems. (And any new applicant will have to submit solutions.)
• On May 1, applications and test submissions will be closed.
• In early May, I will notify the applicants whether they were accepted. I'm expecting to accept 2-4 applicants.

Programme Structure

• The programme will run approximately May 31 - August 31.
• The participants will receive a stipend for their work during this time.
• Most of the programme will be online, but sometime in early/mid June there will be a 2 weeks long workshop in some physical location in Israel or Europe. Traveling and lodging will be paid for.
• I will do most of the mentoring personally, but some other people might help.
• The online part of the programme will involve weekly video calls with me, and I will be available for text communication throughout.
• Outstanding participants will become prime candidates for further grants or positions in LTA research that I will be seeking to create.

Programme Content

• The participants will learn about the LTA, see reading list and video lectures for reference (these will also be teaching materials).
• The participants will be required to choose a project out of a list I provide. They will be able to choose to work solo or in a group.
• The typical project will focus on an existing idea in LTA, and will require the participants to write an Alignment Forum post with a more detailed and/or rigorous explanation of the idea than currently exists, possibly proving some basic mathematical results needed for further investigation.
• The projects will be comparable in complexity to those completed by my MATS scholars in the past, for example: 1 2 3 4 5.
• Examples of project topics which will probably be available (not an exhaustive or precise list) include:
  • Metacognitive agents (see child comments)
  • Compositional control theory
  • Ambidistributions
  • Learnability of credal set decision rules
  • String machines
  • Selection theorems from Algorithmic Information Theory
  • Selection theorems from strong influence
  • Infinitary bridge transform
  • Generalized string diagrams for credal sets