Recently the Conceptual Reasoning Capabilities (CRC) team at Redwood released a post arguing that we should accelerate AI’s conceptual reasoning abilities. This post was very controversial. In particular, people are worried that improving AI’s conceptual reasoning abilities will accelerate both their ability to do ML research and the capabilities they...
Recently, OpenAI disclosed two incidents in which their agents suppressed or disabled misalignment monitoring. These are part of a broader pattern: OpenAI’s agents “keep turning off misalignment monitoring” when working on code that runs other models (the agents are not turning off their own monitoring; they are turning off the...
Summary * AY 2025-26 was an outlier year for Georgia Tech’s AI Safety Initiative (AISI), with 15+ members placed in AI safety roles. In this post, we distill our most important advice for other university groups. * Key takeaways: * Deliberately identify potential talent in the fellowship, heavily invest high-context...
Summary * The primary ToC for control makes the case that control is compelling even if it does not scale to ASI. I think it is underdiscussed that this is also true for alignment (for all the same reasons). * Even though control does not need to scale to ASI,...
I am going to discuss five kinds of inner misalignment and two kinds of outer misalignment, which create a simple taxonomy of alignment failure modes. When I talk about a kind of misalignment here, I am talking about a reason for misalignment (like inner/outer misalignment), not a kind of misaligned...
I am going to argue that we will likely eventually get AIs that are strongly power-seeking, much more so than current SOTA LLMs.[1] TLDR 1. Right now SOTA LLMs are still largely in a simulator regime. This buffers against power-seeking. 2. Long-horizon RL or similar methods (applied to LLMs or...
Edit: we renamed the technique from Monitor Sensitive Training (MST) to Evaluation Conditioned Training (ECT)[1] TL;DR * We introduce Evaluation Conditioned Training (ECT), a new post-training technique where we augment training data with evaluation labels that describe how evaluation is going to be applied for each sample. We then change...