This is a special post for quick takes by Daniel Tan. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I think AIS could benefit from "forward-deployed research engineers".
Consider that:
Within AIS, we often talk about "differentially accelerating safety" (as opposed to capabilities)
Automation can provide extreme uplift (as reported by Boris Cherny, anyway). Also, the ceiling on automation climbs extremely rapidly with time (e.g. METR time horizons) and has not plateaued yet.
However, the benefits have yet to diffuse widely into society at large.
Some of this is irreducible (e.g. some tasks are hard to automate)
But some is reducible (e.g. orgs just don't know how / haven't allocated time etc)
I'd guess there are a bunch of organizations which are doing good work now + have yet to fully benefit from automation, and so could do a lot more good work
So: plausibly a pretty good thing to do is to just hire a large number of research engineers, who have research engineering + automation skills, and "forward deploy" them at orgs which would benefit from being sped up. Alternatively, have consulting services which help orgs implement best practices.
Corollary: It might be a good idea to create new orgs which offer such "forward deployed research engineer" services (AE studio comes to mind, but IMO we could use more)
where would you see these AI safety forward deployed engineers mostly work at?
from your description, I believe they have the highest value in model labs as we are talking about AIS wrt to catastrophic risks, but there aren’t really that many labs — and I don’t think Chinese labs would hire these engineers from US/UK/EU.
I see that a lot of startups are hiring for security research engineers (eg Mirendil that focuses on AI R&D that accelerates science) which I believe that’s having a higher demand at the moment but my understanding is that theres a higher focus on alignment researchers rather than security researchers within LW community (at least based on my read on programs like MATS)
I think forward-deployed research engineers (FDREs) should mainly work with independent nonprofits / academic labs who have a good theory of change but wouldn't have capital to normally attract RE talent by themselves
I think there should be conditions attached to the use of such research engineers, e.g. working towards an agreed agenda, and making sure to publish all findings in an unbiased way.
In practice FDREs should be viewed as another type of "grant", which is somewhat-fungible (but not totally fungible) with monetary grants, and doled out by equival... (read more)