This is an email I wrote to @Buck after he gave a talk about his strategy confusions to a Constellation audience. The talk was about how Redwood's work could have stopped the Hugging Face incident from happening, which would have been very bad for the salience of AI safety, and how he's confused about that. I think this is a very important consideration which should be shared more prominently, but I also think ALL of Redwood's strategic considerations should be shared more prominently. Here's an email I wrote to Buck, lightly edited for clarity.

Hi Buck,

I'm pretty new to the AI safety space. I've loosely followed thing for a few years, but only really sunk my teeth in for about five months. Four months ago, I was mistaken about what leading safety orgs believed, and what their motivations were. For example, I thought that Redwood's control regime was misguided, because it wouldn't scale to ASI. I now believe (as Redwood does) that control is mostly good because it allows us to mine labor from, say, just barely superhuman AIs which could do tons of work to align ASIs. The point of this anecdote is that I think having transparency about your strategy beliefs as an org at the "center" of the AI ecosystem is really important for directing people who are newer to the space to grapple with the right strategy questions (and not jump in to doing useless stuff). I think many people new to the space pursue projects which are unhelpful because they are working under the worldview of "control good" rather than [insert convoluted Redwood worldview which probably endorses some kind of control research but not others]. I largely avoided this by being confused and being more interested/motivated by strategic and abstract questions than by hairy empirical ones. Probably many people who like thinking about hairy empirical problems and not strategy will just get nerd sniped and not do the best things no matter what you do, but we can at least move the space in the right direction. 

Anyways, this summer I noticed I was confused about whether implementing control mechanisms right now is good at all. Warning shots matter so much! And then you gave a talk about this topic which I enjoyed, and my sense in the room was that the talk raised an important question that most people had not deeply considered. I'm very worried about the contribution of "AI safety" towards putting us in a looks good, is bad world [Buck's framing from his talk]. I think you should make your talk into a brief write up and post it somewhere. In general, I think you should more prominently place your strategy considerations on Redwood's website and research. I don't know the best way to do this and I am sure there are significant trade offs I have not considered. But if Redwood isn't transparent about strategy, who will be! Not just transparent somewhere in a blog post from a few months ago, but clearly transparent with a "do not try this at home unless you understand this blog post" disclaimer next to research that you think is probably good for a very specific reason. I would like the AI safety space to be more confused, more correct, and less headstrong. 

Best,

Ben Pomeranz

@Cleo Nardo points out that this also has a lot to do with field building research programs, where people want to start doing research on day three but should probably be considering strategy for a few weeks. He also points out that the control stuff was preceded by posts like “the case for ensuring AIs are controlled,” but I think that these justifications should be featured prominently alongside the research that they justify, because many people will engage with and build off of technical work without looking back in time for relevant blogposts that may precede it. Also, he points out
that Redwood's more recent research direction into conceptual uplift does not come with a matching "the case for conceptual uplift" post.