@Buck was so kind as to discuss with me my confusions about Redwood Research (RR) and their level of transparency. You can read my original comment here. I am relaying Buck's response to the best of my ability, but probably am getting some things wrong.
TLDR of what I said: It seems very easy for people new to AIS to get confused, and do useless things. Maybe RR should be more transparent, and highlight the nonobvious strategic considerations which inform their research agenda. Also, transparency might allow for better external feedback to RR, e.g. about control preventing warning shots.
TLDR of Buck's reply: It's not that bad for people who don't think about strategy to do useless stuff, and RR's overall strategic considerations over time have been somewhat consistent. Those considerations are shared a few paragraphs into blogposts that people who want to understand RR can read past the first paragraph. Also, HF was confusing but not a huge update for RR, which had already considered the warning-shot-reduction externalities of control.
I took notes during our discussion, and below is Buck's reply in more detail, to the best of my understanding. That may not be a very strong understanding and I expect I unintentionally misrepresent Buck at least once:
I'm pretty satisfied by this. I previously overrated RR's strategic uncertainty in the wake of the HF incident, and probably overrated how helpful more external feedback on their strategic thinking would be. Still, I would prefer if RR shared their strategic thinking more broadly than they do now, and think this could be pretty cheap. Specifically, when publishing research it would be pretty easy to highlight the relevant strategy writing which informs it (if it exists, which Buck seemed to say it mostly did). For example, in their 2024 paper AI CONTROL: IMPROVING SAFETY DESPITE INTENTIONAL SUBVERSION—and I don't mean to pick on a paper from two years ago, but I think it's an important example—the authors only mention in section 5.1.2 that they expect control to be useful despite the fact that they do NOT expect control to scale to ASIs:
"Therefore, as more powerful untrusted models are developed, and the gap between these powerful untrusted models and trusted labor grows, AI control will become a decreasingly effective approach to safely using powerful AI. We think control is a valuable research direction anyway, because it can be applied to reduce the risk from using AIs at capabilities levels which we think would otherwise pose a lot of the total risk. Applying control to these powerful AIs might allow for productively and somewhat safely using these AIs for key AI safety applications like further AI control research or alignment research..."
They do not mention these considerations in the abstract, introduction, or conclusion of the paper, nor do they denote where people can go to read about the relevant strategy considerations in further detail.
I would also still like if RR were spicier, and posted their beliefs more often. I see how this could have some important costs in terms of status outside of the AI safety bubble and RR's time. I also think it would have a very nice effect on the Constellation-epistemic-bubble (and relatedly, I broadly agree with the ~40% that I understand of Alex's thoughts here). I'm not sure if this is worthwhile at the margin, because I don't have a good sense of the costs, but Gosh it would be nice.
This was not a direct quote. The note I wrote as Buck was speaking was "the AI companies were doing semi complicated control stuff," but I paraphrased some things as he went. I think the "semi"/"somewhat" here is pretty load bearing. My understanding is that once the labs are putting in even a relatively small effort towards developing good control mechanisms, outsiders can better control-ify them by red teaming, rather than trying to develop control methods themselves to be ported into the labs. Buck didn't say that though, and I'm not sure what their exa... (read more)