I have been getting increasingly active towards understanding some of the programs mentioned, with a view to get into one (or some) of them. Most recently I looked at Algoverse. I was surprised to see it it's basically a pay-to-play program, mostly targeting high-schoolers, where for ~$3k one is given a ~70% probability of publishing at NeuroIPS workshops. It even had unflattering coverage in The Guardian [1]. To me it sounds like college admissions prep industry extending into research credentials, and I can't see how that will improve the quality of NeuroIPS and others.
I'm new to the AI field, so I wonder how much of the off feelings I get are just a different research culture, younger field (??), etc. But this seems a shade too far, and the inclusion on this list makes me wonder about the rest.
And yet, Algoverse does boast about getting their people into frontier labs. So I would love some context: is pay-to-play a thing in AI research? I haven't noticed other programs in the list working in the same way.
[1] https://www.theguardian.com/technology/2025/dec/06/ai-research-papers
Regarding the "rounding error": if I'm counting right, just in the 2025 ER NeurIPS workshop you're listed as co-author for 10 of 223 papers; there might be more, since the Algoverse rules say it's optional to list you as an author. That's 4.5%, hardly a rounding error. Isn't it awkward to agree that something is a problem that should be fixed... while turning it into a business?
Regarding the The Guardian article: I would love to know more about those reasons why it's misleading. Thank you for offering.
Navigating AI safety fellowships and other programs as an early researcher is confusing because there are many of them, and it's not obvious which to prioritize or in what order. This post gives a rough progression path.
For this post, we assume the goal is to get a full-time job as a technical AI safety researcher. Fellowships and other programs help achieve this goal in three ways: 1) upskilling, 2) gaining career capital, and 3) networking in AI safety community.
Let me lay some background on skills, career capital, and networking, and after that, my thoughts on how to navigate AI safety programs.
Background
Skills
We can roughly divide technical AI safety research into two categories – empirical and theoretical. Empirical research is mostly about hands-on experiments with transformers, and theoretical research is essentially mathematical research. The majority (I would estimate >80%) of full-time AI safety researchers do empirical research, and the majority of teams on fellowships do empirical research, so I will concentrate on that.
Empirical research. A non-senior empirical researcher needs two main sets of skills:
You can see fellowships through the lens of upskilling via these two axes. There are two corresponding archetypes of people who participate in fellowships:
Note that if you already have both engineering skill and general research skill – for instance, you are a machine learning researcher with several publications – then you probably should apply only for the top fellowships (like MATS) or straight to the full-time researcher jobs.
Career capital
Fellowships and other programs generate career capital in two ways – output and resume lines, with the first one having much more weight.
Output. The usual output is a write-up, but it can also be a benchmark or a dataset. For write-ups, here is the approximate order of importance:
Many conferences accept two types of papers – for workshops and for the main venue. Workshop papers have a lower bar for results and are easier to get accepted than papers for the main venue, so workshop papers generate less career capital than main venue papers. Order of authors also matters – being the first author generates more career capital.
Resume line. Having AI safety-related lines in your resume helps you get other AI safety positions.
Note that you need career capital not only to land a full-time job, but to progress through different fellowships and programs, from less selective to more selective ones.
Networking
In online fellowships, you will mostly communicate with your mentor and team members. In on-site fellowships, you have a chance to talk to all other mentors, mentees, and potentially other AI safety researchers who co-work in the same place where your fellowship is.
Approximate path through AI safety programs
Level 1. BlueDot
Two BlueDot courses – AGI strategy (5 days) and Technical AI safety (6 days). They give a high-level understanding of the AI Safety area and networking with peers. I do not know the exact acceptance rate, but getting here should be easy.
From the career capital perspective, nearly all other AI safety programs are more selective than BlueDot, and having a BlueDot certificate increases your chances of being accepted.
Level 2. Machine learning upskilling, research upskilling
ARENA (1 month). Covers deep learning fundamentals, transformer architecture, and mechanistic interpretability, all with hands-on programming. Many mentors on fellowship explicitly or implicitly assume that you know material from ARENA.
If you do not have an engineering background, ARENA is your main resource to upskill in engineering. If you have an engineering background but have not worked with transformers, then ARENA is your main resource to upskill in this area.
You can go through ARENA in two ways – either get accepted to an in-person bootcamp, or work through the online material on your own. In the latter case, I recommend joining ARENA’s Slack and taking part in one of the study groups there, which will help you stay motivated.
Other upskilling resources:
Levels 3 and 4. Fellowships
On a fellowship, you and several other fellows form a team mentored by a senior scientist, working together to produce research results. Most fellowships are ~3 months long. There are two types of fellowships
The first type is easier to get into. For instance, chances of getting to SPAR Fall 2025 were ~20% (if you applied to 3-5 projects)[1], while for MATS Summer 2026, the acceptance rate is ~5%[2].
Level 3. Part-time, unpaid, online fellowships
The main goal here is to get a first strong research result, preferably a publication at a top ML conference. There is no guarantee that it will happen on the first try, so you will probably need to participate in several part-time fellowships in a row. The good thing about part-time fellowships is that you can pursue them in parallel with a full-time job.
Here are five such fellowships that I know about.
The other four fellowships are more selective. Here they are in alphabetical order, with brief comments:
Level 4. Full-time, paid, on-site fellowships
These are essentially full-time researcher positions. So if you have been accepted to such a fellowship and have generated strong results during it, then you are ready for researcher positions in AI safety labs.
From anecdotal evidence, the median person accepted to such a fellowship has at least one first-author paper at a main venue of a top ML conference, or equivalent achievement. I do not have the exact acceptance rates for each of these fellowships, but I would estimate them as 3% - 10%.
Here are the fellowships that I know about, in alphabetical order:
There are also two AI safety fellowships from AGI labs:
Miscellaneous
https://sparai.org/advice/, see section "Apply to lots of projects"
https://www.linkedin.com/posts/ryan-kidd-1b0574a3_mats-received-a-record-breaking-2368-applications-share-7423450552828645376-2Sbc/
https://agucova.dev/now/ section "Building Kairos" mentions 400+ SPAR mentees, I guess more than half of them are in technical AI safety
Not for empirical researchers, but I want to mention it. This fellowship is for researchers from other fields to do interdisciplinary research in AI safety.
https://x.com/PingbangHu/status/1980867328163402181