Below, I make the distinction between "MIRI mathematician" and "MIRI engineer". (In my mind I tend to think of these as "researchers" and "engineers", respectively, but I think MIRI calls both of these classes of people "researchers" these days so I want to avoid using "researcher".) Basically I count anyone who has published a paper or post in agent foundations as a mathematician, and everyone else as an engineer. From the current team page, I would classify Nate, Eliezer, Benya, Scott, Sam, Abram, and Tsvi as "MIRI mathematician", and Jesse, Nick, Buck, Ben, James, and Edward as "MIRI engineer". I don't actually know if this is a reasonable classification given that MIRI's recent work isn't public.

As far as I can tell, MIRI has not made any new mathematician hires since mid-2017; see this table which I made, and this blog post which I believe is the last hiring update for mathematicians.

Assuming there are no unannounced hires, the lack of hires can be for two broad reasons:

  • A change in the demand: less need for new people, more selective about who to hire, etc.
  • A change in the supply: fewer people to pick from, drop in quality of people that come into contact, etc.

I think both changes have probably happened, although my guess would be that with the takeoff of the AI safety field, there are now more people to pick from, so the lack of hires is mostly due to a change in the demand.

One way I've been thinking about these changes is to consider the same person attempting to work at MIRI as a mathematician in 2013 vs 2019. In particular, I've been thinking about Nate Soares, who was hired as a research fellow in 2014. To give my own summary, Nate was a full-time Google engineer who had taken multi-variable calculus and real analysis in college (in addition to the mathematics appearing in his CS degree), but otherwise with no experience learning or doing research in math. He discovered LessWrong probably in early or mid 2013, and came around to MIRI's worldviews by July 2013. He then spent five months (August to December) learning basic category theory, set theory, model theory, computability and logic, and provability logic, then attended a MIRI workshop, studied some more math, and joined MIRI as a research associate (January 2014) and then as a research fellow (April 2014). Nate might be an outlier in his studying/research abilities, so might be a bad example to use here, but he is the most transparent example, having blogged about his experiences. See The mechanics of my recent productivity, On saving the world, and his other LessWrong posts for more information.

I am wondering what the corresponding trajectory would look like for a Nate Soares who discovered LessWrong in 2019. (This version of Nate would be six years younger. Also discovering LessWrong later means a lower quality person in expectation, even adjusting for age, so we can imagine something magically prevented him from discovering it.) Would he have been hired? If so, how long would it have taken him? If not, what would have happened to him?

I've brainstormed some potential differences between 2013 and 2019. Note that I'm not saying these are the only reasons or that I think any particular reason listed is likely (I'm asking this question because I'm not sure what's actually the case).

  • Takeoff of the field in general (see Timeline of AI safety, Timeline of OpenAI, Timeline of MIRI, Timeline of FHI, and Timeline of CHAI for some relevant dates)
    • Now there are more AI safety organizations, who could be competing for new talent. In other words, a 2019 version of Nate could be hired by OpenAI instead of MIRI.
    • More people interested in entering the field. I think this would mean more hires, so doesn't explain the lack of hires.
    • Position of MIRI relative to the AI safety field. In 2013 MIRI represented the AI safety field as a whole, but in 2019 MIRI represents a particular view within AI safety. There are now more filters for a potential hire, where they must not only agree with AI safety as a cause area, but also with MIRI's specific positions within that cause area.
  • Less openness at MIRI (e.g. nondisclosed-by-default policy, fewer MIRI people who are active on public forums). A 2019 version of Nate would have a harder time learning about and contributing to the cutting edge of MIRI research, and interacting with MIRI researchers.
  • Emergence of MIRI's new research directions. This might mean MIRI is more focused on hiring engineers, so a 2019 version of Nate might have become an engineer at MIRI instead of a mathematician.
  • Lack of MIRI workshops (I've heard the growing size of MIRI's research team as a reason for this, i.e. workshops are less necessary because there are more people already at MIRI to bounce ideas off of). A 2019 version of Nate wouldn't be able to attend a workshop, unlike the 2013 version.
  • Emergence of programs like MSFP, internships, AIRCS, and retraining project. MSFP in particular seems to serve a similar recruiting function for mathematicians as workshops, but I'm not clear on the differences from a hiring viewpoint.
  • Bigger research team at MIRI. Maybe the "basic research positions" at MIRI have been filled, so MIRI can be more selective about the expertise new hires have, or their level of capability. Another idea is that MIRI research is not very parallelizable, so the value of the marginal hire goes down quickly as a function of number of researchers.
  • Funding. MIRI seems to have way more funding now, so this doesn't explain the lack of hires.

I am interested in thoughts on the specific questions I've raised, thoughts on which of the reasons brainstormed above are most likely, and any other thoughts people have on MIRI hiring practices in 2013 vs 2019.

Thanks to Matthew Barnett, Louis Francini, and Vipul Naik for feedback on this question.

New Answer
New Comment

2 Answers sorted by

110

I would hope that someone from MIRI would chime in on this thread. The OP presumably might want to join MIRI, and there are others who might want to as well. Is there anyone at MIRI who is thinking about onboarding new talent in light of the issues raised by OP?

Stephen James

60

I have wondered this exact same thing myself, having discovered LessWrong in 2018 and Nate's story very soon thereafter. We have a similar-enough background, though I'm missing the basic analysis course in university. Your analysis lines up with almost everything I have gleaned over the last year, when things have seemed much quieter.

My effective conclusion - in absence of more information - is that MIRI is "full" like one is full from a meal. It would be nice to have more people on the mathematical side of things, but it's not going to help for a little while; you noticed this in the lack of workshops these days.

My resolution has been to get a degree in mathematics, as to preclude future missed research opportunities. We haven't automated mathematicians just yet.

1 comment, sorted by Click to highlight new comments since:

I partially see it as a question of volume/consistency than just raw talent, though a bigger pipeline on raw talent is obviously still an obstacle. I think there are quite a few people who can put out relevant work occasionally, and very few who can do it often enough that it makes sense to be full time.