This is a special post for quick takes by DW11. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
In modern LLM training cycles, at what point is training for alignment carried out? Is it a continuous process from pre-training onwards (or during pre-training?)
Just thinking about this in light of OAI seemingly asking an internal model that hasn't been fully trained to solve an extremely hard task and giving it practically unlimited compute to do so.