This is a special post for quick takes by Daniel Tan. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Various theories of change of AI safety work seem to route through "convince lab employees that X is a problem", e.g. so they can then pressure leadership / champion this problem internally.
I think this theory of change is overrated. It probably works best for some small set of X that is very relevant to what lab employees are already thinking about, but will not work for X that falls outside this category (which is plausibly a lot of what's important!)
More generally my mental model of lab employees is:
they are overworked, e.g. spending 60+ hours / week on their current project(s). The upshot is that some of this time can be allocated to your choice of X if it's clearly relevant somehow. But the downside is that they have little extra time to be thinking about X that has nothing to do with their current work.[1]
on topics which seem important (to have an opinion about) but which are not their area of expertise, they are likely to defer to internal colleagues rather than external voices. This is for various reasons e.g. internal colleagues having internal context, having been vetted by process of hiring, or simply being more personally close. So your opinion on X will probably be overridden by [internal colleague]'s opinion.
This phenomenon strikes me as not-too-dissimilar from historical examples of "disempowerment by overwork", e.g. when Mao spearheaded the transformation of Chinese intellectual elites into workers / peasants with gruelling work details
Some examples of X that seem to have been successful are AI control, monitoring, and red-teaming of such. (“Successful” in that these efforts got labs to care about X more, no claim about whether they were net positive for safety)
Probably many examples of X that have not been successful.
Overrated compared to what? We're in a bit of a sticky situation here; there are no ideal options.
I've heard you advocate producing better evidence of misalignment. It seems to me we've got plenty, and communicating it to people who make decisions is the bigger challenge. Not that such work couldn't be valuable on the margins and more if it's really well-done, of course.
Just to put my cards on the table, I think that communicating about x-risk to everyone, public, politicians, and lab employees alike is high-priority. Sure there are difficulties. Alongside that I'd rank thinking ahead on alignment (labs probably don't do it much for the reasons you mention - they're very busy) and doing big-picture prioritization (like this discussion but in vastly more depth) are the biggest priorities.
2Daniel Tan
Oh I agree communicating is high priority. There’s a long list of people (most of whom you mentioned) that are higher priority than lab employees though. (Historically true and will become less true as people update)
I also think it‘s not bad to be communicating to lab employees. However, it‘s unclear to me how successful this is. I don’t see that many successful historical examples of this having convinced them to do something they weren’t already thinking about doing (AI control being a notable counterexample)
I do think it‘s not ideal to be an org working with a lab in such a way that prevents you from being able to talk to people outside the lab candidly.
So tldr if your theory of change is “gather evidence to persuade the lab employees” then most of the time I think it’s better to replace that with “gather evidence to persuade the rest of the world so that they put societal / legal pressure on labs”