This is a special post for quick takes by Sam Marks. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
We are moving rapidly from a world where people deploy manifestly unaligned models (where even talking about alignment barely makes sense) to people deploying models which are misaligned because (i) humans make mistakes in evaluation, (ii) there are high-stakes decisions so we can't rely on average-case performance.
This seems like a good thing to do if you want to move on to research addressing the problems in RLHF: (i) improving the quality of the evaluations (e.g. by using AI assistance), and (ii) handling high-stakes objective misgeneralization (e.g. by adversarial training).
In addition to "doing the basic thing before the more complicated thing intended to address its failures," it's also the case that RLHF is a building block in the more complicated things.
I think that (a) there is a good chance that these boring approaches will work well enough to buy (a significant amount) time for humans or superhuman AIs to make progress on alignment research or coordination, (b) when they fail, there is a good chance that their failures can be productively studied and addressed.
Overall it seems to me like the story here is reasonably good and has worked out reasonably well in practice. I think RLHF is being adopted more quickly than it otherwise would, and plenty of follow-up work is being done. I think many people in labs have a better understanding of what the remaining problems in alignment are; as a result they are significantly more likely to work productively on those problems themselves or to recognize and adopt solutions from elsewhere.