This is a special post for quick takes by Jacob Brinton. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
The central talent problem in AI safety is not “making the number of full-time people in AI safety go up” but “supporting niche talent needs”. It is becoming clearer what work lies on the critical path, and the majority of people in AI safety are not doing it.
Rationale:
1. The situation is urgent. Note that this is not necessarily the same as timelines being short, but it is similar. The political winds are shifting (perhaps in our favor) while AI becomes a partisan issue and labs are getting closer to RSI.[1]
2. There is a massive misbalance in the number of people working in certain subfields relative to their importance. The number of aligned people working in comms is probably in the double digits. There is at least 10x that amount of people working on technical research, and I don’t think that comms is less than 10% as important as technical research. Of course, not all of the technical researchers would be great at comms, but I find it hard to believe that less than 10% of them would be decent, especially if put on the right team.
3. We don’t need safety engineers at labs papering over cracks.[2] However, the fact that they have crack-papering experience is valuable in speaking credibly about the risks, whether to DC people or mass audiences.
4. I suspect that many current employees of METR or Redwood are smart enough that they could tackle a new vertical (e.g. operationalizing what a kill switch looks like) relatively soon, but I find it harder to believe someone new to AI safety could do the same successfully, because they lack connections and general AI risk know-how.
5. The first comms and generalist fellowships (mostly) happened this year. There have been over 10 iterations of MATS, which has churned out great results, but there are a ton more MATS alumni than Generator alumni.
Counterarguments:
1. It’s good to not put all your eggs in one basket. My counter-counterargument is that even if all the fieldbuilding orgs decided tomorrow to spend twice as much time on talent allocation as capacity building, I still would not expect us to overcorrect within the next 6 months.
2. There are new streams of talent available to us that weren’t there pre-Coxon/HF, and we should take advantage of that. I agree with this, although I don’t see enough people working on this either.
3. It’s easier to hill-climb on “# of people in AI safety”, and really make your pipelines efficient. Building a lot of ad-hoc transfer pipelines takes time.
Weak version of my argument: Even if you disagree with the above, I don’t think it’s a stretch to say that fieldbuilding orgs should make large changes to their prioritization. I think it’s rather clear that getting a single person who has 10+ years of DC experience and connections into the field would be a couple OOMs more impactful than a junior technical researcher.
See METR’s report today: since October 2025, benchmark scores for Anthropic’s internal models have grown at about twice their historical rate (1.9x, 95% CI 1.5–2.8x).
Note I’m not arguing that we need 0, just less on the margin.