My guess is that there are at least a few capabilities gain researchers who're concerned about the impact their work will have. My guess is that at least a few of these would like to leave, but haven't. 

My question is: where are these people? What's stopping them from leaving? And how can I help?

  • How much of it is finance? Capabilities gain pays well. How many researchers are trapped by their ~million dollar a year salary?
  • How much of it is just inertia? Many people think that if someone wanted to leave, they already would have. But trivial costs are not trivial. People delay leaving a job all the time. Some of them are quite likely working in capabilities gain research.
  • How much of it is just. Uncertainty about what else to do. Concerns about whether to it's better to leave, or to try and steward the ship from the inside. Many, many other things. How many of these other myriad concerns can be resolved? 

My suspicion is that a lot of work can be done to simply help people leave the field. In principle: a low friction scholarship can be presented on a silver platter to those wanting to transition away. Potentially useful information, and personalised support can be provided to help people interested in leaving. (AI safety support is trying to work with those interested in entering AI safety; something like that, but for leaving capabilities gain.) A simple conversation can be started, which may prompt people to address that voice of concern in the corner of their mind more head on. 


There is a lot of uncertainty about what low hanging fruit exist in simply helping researchers leave. Whether these fruit exist at all. How many. Are they poisonous. 


The first step in resolving this uncertainty is to just look for some concerned researchers, and talk to them. 


So, dear concerned capabilities gain researcher. 

What's keeping you from leaving? 

(DM me if you wish for privacy)

New Answer
New Comment

1 Answers sorted by


May 13, 2022


If they leave then only who does not care remains... 

Not everyone concerned about safety is looking to leave. The concerned have three options: stay and try to steer towards safety, continue moving on the current trajectory, or just leave. Helping some of those who’ve changed their mind about capabilities gain actually get out is only a net negative if those people staying in the field would’ve changed the trajectory of the field. I simply don’t think that everyone should try help by staying and trying to change. There is absolutely room for people to help by just leaving, and reducing the amount of work goi... (read more)

What's stopping the companies from hiring a new researcher? People are queueing for tech jobs.
Researchers aren't exactly fungible; replacing a skilled researcher with a new hire would still slow down progress. given how many people want to help, but have no idea how to help, this is a gap in the market worth filling.