I get the feeling that for AI safety, some people believe that it's crucially important to be an expert in a whole bunch of fields of math in order to make any progress. In the past I took this advice and tried to deeply study computability theory, set theory, type theory -- with the hopes of it someday giving me greater insight into AI safety.

Now, I think I was taking a wrong approach. To be fair, I still think being an expert in a whole bunch of fields of math is probably useful, especially if you want very strong abilities to reason about complicated systems. But, my model for the way I frame my learning is much different now.

I think my main model which describes my current perspective is that I think employing a lazy style of learning is superior for AI safety work. Lazy is meant in the computer science sense of only learning something when it seems like you need to know it in order to understand something important. I will contrast this with the model that one should learn a set of solid foundations first before going any further.

Obviously neither model can be absolutely correct in an extreme sense. I don't, as a silly example, think that people who can't do basic arithmetic should go into AI safety before building a foundation in math. And on the other side of the spectrum, I think it would be absurd to think that one should become a world renowned mathematician before reading their first AI safety paper. That said, even though both models are wrong, I think my current preference is for the lazy model rather than the foundation model.

Here are some points in favor of both, informed by my first-person experience.

Points in favor of the foundations model:

• If you don't have solid foundations in mathematics, you may not even be aware of things that you are missing.
• Having solid foundations in mathematics will help you to think rigorously about things rather than having a vague non-reductionistic view of AI concepts.
• Subpoint: MIRI work is motivated by coming up with new mathematics that can describe error-tolerant agents without relying on fuzzy statements like "machine learning relies on heuristics so we need to study heuristics rather than hard math to do alignment."
• We should try to learn the math that will be useful for AI safety in the future, rather than what is being used for machine learning papers right now. If your view of AI is that it is at least a few decades away, then it's possible that learning the foundations of mathematics will be more robustly useful no matter where the field shifts.

Points in favor of the lazy model:

• Time is limited and it usually takes several years to become proficient in the foundations of mathematics. This is time that could have been spent reading actual research directly related to AI safety.
• The lazy model is better for my motivation, since it makes me feel like I am actually learning about what's important, rather than doing homework.
• Learning foundational math often looks a lot like just taking a shotgun and learning everything that seems vaguely relevant to agent foundations. Unless you have a very strong passion for this type of mathematics, it would seem outright strange that this type of learning is fun.
• It's not clear that the MIRI approach is correct. I don't have a strong opinion on this, however
• Even if the MIRI approach was correct, I don't think it's my comparative advantage to do foundational mathematics.
• The lazy model will naturally force you to learn the things that are actually relevant, as measured by how much you come in contact with them. By contrast, the foundational model forces you to learn things which might not be relevant at all. Obviously, we won't know what is and isn't relevant beforehand, but I currently err on the side of saying that some things won't be relevant if they don't have a current direct input to machine learning.
• Even if AI is many decades away, machine learning has been around for a long time, and it seems like the math useful for machine learning hasn't changed much. So, it seems like a safe bet that foundational math won't be relevant for understanding normal machine learning research any time soon.