Authors: Kyle O’Brien¹, Spencer Kitts², Cameron Tice¹, Alek Westover² ¹Geodesic Research, ²Redwood Research Figure 1: We pretrained LLMs from scratch with and without filtering subversion-relevant information, finding that subversion-relevant knowledge is significantly diminished while retaining general ML knowledge. TLDR: Filtering subversion-relevant information — subversion strategies, information about subversion defences, and...
How philanthropic organisations can help close the resource gap between frontier labs and independent AI safety research. This post draws on Geodesic Research's experience deploying philanthropic funding in support of a compute-heavy research agenda. Over the past six months, through this procurement campaign, we have identified non-obvious bottlenecks that, if...
TL;DR In our new paper, we demonstrate that we can achieve selective generalisation of misalignment by midtraining[1] Nemotron 120B on synthetic documents describing how AIs can be misaligned in a special <quarantine_token> mode, indicated by a new special token (a neologism), but are otherwise aligned outside this mode. We find...
We're a Cambridge, UK-based AI safety organisation that’s asking: how can we build the most robust alignment initialisations for capable LLMs? We’re one of the few non-profit organisations positioned to answer this question empirically. We have the engineering experience, and now the compute, to conduct data intensive interventions across the...
TL;DR LLMs pretrained on data about misaligned AIs themselves become less aligned. Luckily, pretraining LLMs with synthetic data about good AIs helps them become more aligned. These alignment priors persist through post-training, providing alignment-in-depth. We recommend labs pretrain for alignment, just as they do for capabilities. Website: alignmentpretraining.ai Us: geodesicresearch.org...
Apply for the ERA:AI Fellowship! We are now accepting applications for our 8-week (February 2nd - March 27th), fully-funded, research program on mitigating catastrophic risks from advanced AI. The program will be held in-person in Cambridge, UK. Deadline: November 3rd, 2025. → Apply Now: https://airtable.com/app8tdE8VUOAztk5z/pagzqVD9eKCav80vq/form ERA fellows tackle some of...