Top postsTop post
Thanks to Benjamin Schmidt and Luke McNally for the discussions around this topic and their feedback on an early draft of this article! Gradual disempowerment is one of the central failure modes of advanced AI. The idea, developed by Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, and...
Anthropic’s most powerful model, Claude Mythos Preview, has alarmed and excited many people, especially given its cybersecurity capabilities[1]. But what concerns me most is something else: in two months, Anthropic appears to have made as much progress in AI development as would normally take half a year. What if this...
Some argue that AI progress will speed up as AIs help with their own development. Some argue that we will hit a wall. Will progress be smooth, or punctuated by sudden leaps? Using the length of tasks that AIs can complete—their time horizon—as a measure of their general capability, this...
This article aggregates virtually everything currently known about AI scheming, then builds toward an informed forecast. How to read this article: After reading the introduction to understand the article’s scope and structure, I recommend moving directly to the Overview and Forecast sections, and read the background sections as needed for...
Currently, there is minimal regulation regarding the development and use of highly powerful AI systems, like AGI or superintelligence. But even if there were stronger rules or international agreements, could they be enforced? Can you prevent AI companies from conducting secret research and training runs? Can you stop terrorists from...
Are current AIs “aligned”? When interacting with today’s most powerful AIs, they don’t appear very misaligned—they usually refuse to provide dangerous information, use a polite tone, and are generally very helpful (unless intentionally jailbroken or put in contrived setups). Some see this as a hopeful sign: maybe alignment isn’t that...