Roadmap through AI safety programs for early-career technical researchers

Mikhail Mironov

Navigating AI safety fellowships and other programs as an early researcher is confusing because there are many of them, and it's not obvious which to prioritize or in what order. This post gives a rough progression path.

For this post, we assume the goal is to get a full-time job as a technical AI safety researcher. Fellowships and other programs help achieve this goal in three ways: 1) upskilling, 2) gaining career capital, and 3) networking in AI safety community.

Let me lay some background on skills, career capital, and networking, and after that, my thoughts on how to navigate AI safety programs.

Background

Skills

We can roughly divide technical AI safety research into two categories – empirical and theoretical. Empirical research is mostly about hands-on experiments with transformers, and theoretical research is essentially mathematical research. The majority (I would estimate >80%) of full-time AI safety researchers do empirical research, and the majority of teams on fellowships do empirical research, so I will concentrate on that.

Empirical research. A non-senior empirical researcher needs two main sets of skills:

General research – ability to understand research papers, generate interesting questions, design experiments to answer these questions, interpret the results, notice contradictions and missing parts.
Engineering – ability to write code that works, conduct experiments, set infrastructure for these experiments.

You can see fellowships through the lens of upskilling via these two axes. There are two corresponding archetypes of people who participate in fellowships:

Researchers from other science areas. They have several years of experience researching physics/math/cognitive science or other sciences, but have little coding experience. They focus on improving their engineering ability with transformers.
Engineers without research experience. They have several years of software development experience. They either already worked with transformers or quickly learn transformer-related engineering. They focus on improving research skills.

Note that if you already have both engineering skill and general research skill – for instance, you are a machine learning researcher with several publications – then you probably should apply only for the top fellowships (like MATS) or straight to the full-time researcher jobs.

Career capital

Fellowships and other programs generate career capital in two ways – output and resume lines, with the first one having much more weight.

Output. The usual output is a write-up, but it can also be a benchmark or a dataset. For write-ups, here is the approximate order of importance:

LessWrong post or arXiv preprint
Accepted publication to ML conference
Accepted publication to top ML conference (like ICML, ICLR, NeurIPS)

Many conferences accept two types of papers – for workshops and for the main venue. Workshop papers have a lower bar for results and are easier to get accepted than papers for the main venue, so workshop papers generate less career capital than main venue papers. Order of authors also matters – being the first author generates more career capital.

Resume line. Having AI safety-related lines in your resume helps you get other AI safety positions.

Note that you need career capital not only to land a full-time job, but to progress through different fellowships and programs, from less selective to more selective ones.

Networking

In online fellowships, you will mostly communicate with your mentor and team members. In on-site fellowships, you have a chance to talk to all other mentors, mentees, and potentially other AI safety researchers who co-work in the same place where your fellowship is.

Approximate path through AI safety programs

Level 1. BlueDot

Two BlueDot courses – AGI strategy (5 days) and Technical AI safety (6 days). They give a high-level understanding of the AI Safety area and networking with peers. I do not know the exact acceptance rate, but getting here should be easy.

From the career capital perspective, nearly all other AI safety programs are more selective than BlueDot, and having a BlueDot certificate increases your chances of being accepted.

Level 2. Machine learning upskilling, research upskilling

ARENA (1 month). Covers deep learning fundamentals, transformer architecture, and mechanistic interpretability, all with hands-on programming. Many mentors on fellowship explicitly or implicitly assume that you know material from ARENA.

If you do not have an engineering background, ARENA is your main resource to upskill in engineering. If you have an engineering background but have not worked with transformers, then ARENA is your main resource to upskill in this area.

You can go through ARENA in two ways – either get accepted to an in-person bootcamp, or work through the online material on your own. In the latter case, I recommend joining ARENA’s Slack and taking part in one of the study groups there, which will help you stay motivated.

Other upskilling resources:

Apart hackathon (3 days) – online hackathon, short intense version of the AI Safety research process (can lead to Apart Fellowship, more on this later).
BlueDot Technical AI Safety Project Sprint (30 hours) – online, short intense version of the research process.
ML4Good (7 days) - in-person bootcamp, more entry-level and less demanding than ARENA bootcamp.

Levels 3 and 4. Fellowships

On a fellowship, you and several other fellows form a team mentored by a senior scientist, working together to produce research results. Most fellowships are ~3 months long. There are two types of fellowships

Part-time, unpaid, online fellowships (like SPAR)
Full-time, paid, on-site fellowships (like MATS)

The first type is easier to get into. For instance, chances of getting to SPAR Fall 2025 were ~20% (if you applied to 3-5 projects)^[1], while for MATS Summer 2026, the acceptance rate is ~5%^[2].

Level 3. Part-time, unpaid, online fellowships

The main goal here is to get a first strong research result, preferably a publication at a top ML conference. There is no guarantee that it will happen on the first try, so you will probably need to participate in several part-time fellowships in a row. The good thing about part-time fellowships is that you can pursue them in parallel with a full-time job.

Here are five such fellowships that I know about.

SPAR – the largest number of participants (>200)^[3]. As mentioned above, SPAR has a ~20% acceptance rate. SPAR is a default entry point, so I mention it first.

The other four fellowships are more selective. Here they are in alphabetical order, with brief comments:

AI Safety Camp (AISC) – the oldest and most established.
Algoverse AI safety – has AI research fellowships from 2023, started AI Safety branch in Winter 2025-2026, so they are very new.
Apart Fellowship – they have a bit of a different structure from ordinary 3-month fellowships. Firstly, you pass a 3-day hackathon (I mentioned it in Level 2), after that you can get to a 4-6 week Studio (5%-20% of the submissions), and after that you can get to a 4-6 months fellowship (40% of Studio participants)
MARS – has its first full-time one-week onsite, which is good for networking.

Level 4. Full-time, paid, on-site fellowships

These are essentially full-time researcher positions. So if you have been accepted to such a fellowship and have generated strong results during it, then you are ready for researcher positions in AI safety labs.

From anecdotal evidence, the median person accepted to such a fellowship has at least one first-author paper at a main venue of a top ML conference, or equivalent achievement. I do not have the exact acceptance rates for each of these fellowships, but I would estimate them as 3% - 10%.

Here are the fellowships that I know about, in alphabetical order:

There are also two AI safety fellowships from AGI labs:

Anthropic Fellows Program for AI safety research – in Fall 2025 cohort, there were 32 participants from 2000+ applications^[5], which is at most 1.6% acceptance rate.
OpenAI Safety Fellowship – new, announced April 2026.

Miscellaneous

I recommend reading this post by Georg Lange on what application reviewers actually look for in AI safety fellowship applications.
If you are considering leaving your current job to focus on a career transition to AI safety full-time, apply for career transition grants, like Career development and transition funding from Coefficient Giving or Career transition grant from BlueDot. In the optimistic case, transitioning to AI safety takes several months; in the median case, it takes a year or more.
To explore all AI safety fellowships and programs, go to https://aisafety.com/map and either look through them in the “Training town” part of the map, or as cards (filter “Training and education”).

^{^}
https://sparai.org/advice/, see section "Apply to lots of projects"
^{^}
https://www.linkedin.com/posts/ryan-kidd-1b0574a3_mats-received-a-record-breaking-2368-applications-share-7423450552828645376-2Sbc/
^{^}
https://agucova.dev/now/ section "Building Kairos" mentions 400+ SPAR mentees, I guess more than half of them are in technical AI safety
^{^}
Not for empirical researchers, but I want to mention it. This fellowship is for researchers from other fields to do interdisciplinary research in AI safety.
^{^}
https://x.com/PingbangHu/status/1980867328163402181

There's also CBAI for fellowships

I have been getting increasingly active towards understanding some of the programs mentioned, with a view to get into one (or some) of them. Most recently I looked at Algoverse. I was surprised to see it it's basically a pay-to-play program, mostly targeting high-schoolers, where for ~$3k one is given a ~70% probability of publishing at NeuroIPS workshops. It even had unflattering coverage in The Guardian [1]. To me it sounds like college admissions prep industry extending into research credentials, and I can't see how that will improve the quality of NeuroIPS and others.

I'm new to the AI field, so I wonder how much of the off feelings I get are just a different research culture, younger field (??), etc. But this seems a shade too far, and the inclusion on this list makes me wonder about the rest.

And yet, Algoverse does boast about getting their people into frontier labs. So I would love some context: is pay-to-play a thing in AI research? I haven't noticed other programs in the list working in the same way.

[1] https://www.theguardian.com/technology/2025/dec/06/ai-research-papers

Hey, I'm the founder of Algoverse. There are two similar but different things here: our main AI research program, which is tuition-funded, and our AI Safety Fellowship, which is free for participants and funded by philanthropic organizations. The one on the roadmap is actually the free fellowship, but I'll address the paid model, since that seems to be the underlying concern.

On the tuition-funded program, the reason we charge tuition is that we are providing structured mentorship, advising, and compute. This is loosely analogous to why a Master's program costs tuition, but at a smaller scale. A lot of students who are interested in AI research do not have realistic access to university labs, especially high-school students, students at non-target schools, students at universities where AI labs are extremely competitive, and people from non-traditional backgrounds. Our goal is to give some of those students a real path into research rather than have access depend almost entirely on being at the right school, in the right geography, with the right informal network.

We also take financial accessibility seriously, so we have intentionally priced our program at a fraction of other research programs in the space (e.g Veritas AI) and offer substantial financial aid scholarships. We've helped numerous talented students from under-resourced countries like Egypt, Nepal, and Ethiopia, get opportunities in AI research that they would not have otherwise.

It’s also worth mentioning the baseline accessibility problem in AI research before our existence. From what I saw empirically back when I was at Berkeley, the students who got into the very limited research opportunities were disproportionately from financially privileged backgrounds, e.g private schools like Harker, Bellarmine, Exeter, and a strong concentration around SF Bay Area. I don't know how to best quantify our impact on the full income distribution, but I'm confident that we've helped a significant number of students at the least-privileged end of the distribution get opportunities that they wouldn't have had otherwise.

On venue quality: I agree this is a real field-wide issue. The recent explosion in submissions is enormous across the field. Our papers are a rounding error within it, which is also one of the reasons why I think the Guardian article's narrative was misleading, among other reasons (happy to go deeper on that if there is interest). The bar for acceptance for our submissions was the same as the bar for everyone else. We do not control the acceptance bar, and I would be happy to see workshops and conferences raise standards across the board.

It is also worth noting that a large fraction of our paid-program research is still AI safety-related. Many students come in broadly interested in AI/ML, and we are often able to direct that interest toward interpretability, evals, and other safety-adjacent areas rather than pure capabilities work. I see this as a big win based on my beliefs on AI, which I think are pretty aligned with a lot of people here.

As for whether to trust the rest of the programs on the list in the original post, FWIW I can confirm from my experience in the field that this is an accurate list.

Regarding the "rounding error": if I'm counting right, just in the 2025 ER NeurIPS workshop you're listed as co-author for 10 of 223 papers; there might be more, since the Algoverse rules say it's optional to list you as an author. That's 4.5%, hardly a rounding error. Isn't it awkward to agree that something is a problem that should be fixed... while turning it into a business?

Regarding the The Guardian article: I would love to know more about those reasons why it's misleading. Thank you for offering.

Algoverse has
- AI Research Program, which is indeed paid for by participants. This gives me off feelings too.
- AI Safety Fellowship, which is funded by Open Philanthropy (Coefficient Giving), and is free for participants, so I decided to include it in the post.

All other AI safety programs and fellowships in the post are also free for participants.

If one program gives off feelings, shouldn't be the whole provider be considered suspicious?

Personally, it makes me reconsider everything connected. Is NeuroIPS' entry bar so low or so gameable? Is OP/CG paying an icky provider? How Efficient is that?

In fact, shouldn't there be some documentation about the expectations / results of Coefficient Giving funding Algoverse? CG's website doesn't return anything. Neither does Google.

The more I (try to) look into this, the worse it gets.