We’ve just published an updated version of Alignment Pretraining to arXiv, with a couple of important changes:

1. We’ve weakened our claims around tampering and alignment elasticity. Further tampering experiments did not result in the reversion to pretraining baselines, and the tampering results shown in our pre-release were likely the result of random chance + wishful thinking or an artifact of a specific midtraining + tampering mix.
2. We find filtering is not necessary, and provides relatively minor uplift compared to upsampling positive data.
3. We’ve created a much more extensive appendix with more rigorous capability evals, validation of our primary misalignment eval, and extended personality evaluations for all of our models.
4. We added an overview of how our models respond to emergent misalignment training, and show that our models show rates of emergent misalignment similar to those of controls.
5. We’ve also added a more detailed section on the comparison of positive upsampling during end-to-end pretraining, midtraining only, or at the end of midtraining via continual fine-tuning. We find largely positive results for using less data and conducting positive pretraining at the end of midtraining.

Thanks for all of the thoughtful comments during our community review phase. This allowed for quick iteration on our end, and ultimately producing a paper we’re confident in distributing widely. It looks like Alignment Pretraining is fairly low-hanging fruit that is not currently implemented in most labs’ safety pipeline.  You can see our post on X for additional commentary.