This is a special post for quick takes by sjadler. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I feel like there should be a concept handle like 'realigning' a model.
'Changing an existing misaligned model into one that is aligned' might be different than just aligning it from the start, and I want a term that refers to the former, but I'm not sure of one.
Apparently OpenAI used the term "emergent re-alignment" here, so maybe folks already think of this concept as 'realignment' after all?
I also want there to be a concept like 'misalignment remediation' (but less wordy?) that could apply to a broader suite of interventions, which is not just about 'realigning' a particular model, but also includes eg rolling back to an earlier point in training and starting again from there. That is, "once you have identified misalignment, what do you do to increase the chance that your next model is aligned?"