This is a linkpost for https://rohinshah.com/faq-career-advice-for-ai-alignment-researchers/
I still don’t know much about GANs, Gaussian processes, Bayesian neural nets, and convex optimization.
FWIW, I'd recommend investing the time to learn about convex optimization even if you don't think you need it yet. Unlike the other topics on this list, the ideas from convex optimization are relevant to a wide variety of things the name does not suggest. Some examples:
Personally, I consider convex optimization ideas fundamentally useful for thinking in a similar way to linear algebra or multivariate calculus.
(Note: I first learned the topic mainly from Boyd's lectures; he's unusually good at conveying many useful intuitions in those talks, so you might get less value from other lecturers or the canonical convex optimization book.)
Yeah that's definitely the one on the list that I think would be most useful.
I may also be understating how much I know about it; I've picked up some over time, e.g. linear programming, minimax, some kinds of duality, mirror descent, Newton's method.
To quote Andrew Critch:
I get a lot of emails from folks with strong math backgrounds (mostly, PhD students in math at top schools) who are looking to transition to working on AI alignment / AI x-risk. There are now too many people “considering” transitioning into this field, and not enough people actually working in it, for me, or most of my colleagues at Stuart Russell’s Center for Human Compatible AI (CHAI), to offer personalized mentorship to everyone who contacts us with these qualifications.
From math grad school to AI alignment, Andrew Critch
I’m pretty sure he wrote that at least 4 years ago (2016 or earlier). The field has grown enormously since then, but so has the number of people considering it as a research area. So far, I’ve tried to give at least 10 minutes of my time to anyone who emails me with questions; that probably won’t be sustainable for much longer. So now I’m answering the questions I get most frequently. I hope to keep this up to date, but no promises.
Usually, I write a blog post when I think I have something important and novel to say, that I am relatively confident in. That’s not the case for this post. This time, I’m taking all the questions that I frequently get and writing down what I’d say in response. Often, this is (a) not that different from what other people would say, and (b) not something I’m very confident in. Take this with more grains of salt than usual.
Thanks to Neel Nanda, Nandi Schoots, and others who wish to remain anonymous for contributing summaries of conversations.
See the linked post for the FAQ; which will hopefully be kept up to date over time.