Currently, there is minimal regulation regarding the development and use of highly powerful AI systems, like AGI or superintelligence. But even if there were stronger rules or international agreements, could they be enforced? Can you prevent AI companies from conducting secret research and training runs? Can you stop terrorists from...
Are current AIs “aligned”? When interacting with today’s most powerful AIs, they don’t appear very misaligned—they usually refuse to provide dangerous information, use a polite tone, and are generally very helpful (unless intentionally jailbroken or put in contrived setups). Some see this as a hopeful sign: maybe alignment isn’t that...
This post investigates emerging biorisk capabilities and when AI systems may cross important risk thresholds. It lays the groundwork for a broader risk forecast (currently in progress) on the annual likelihood of AI-enabled biological catastrophes. For now, the analysis focuses on general-purpose AIs such as large language models (LLMs). A...
My AI forecasting work aims to improve our understanding of the future so we can prepare for it and influence it in positive directions. Yet one problem remains: how do you turn foresight into action? I’m not sure, but I have some thoughts about learning the required skills. Say you...
TL;DR: This post examines how rapidly AI forecasting ability is improving by analyzing results from ForecastBench, Metaculus AI tournaments, and various prediction platforms. While AI forecasting is steadily improving, there remains significant uncertainty around the timeline for matching top human forecasters—it might be very soon or take several years. ForecastBench...
A thoughtful post by Anton Leicht, “Powerless Predictions,” notes that forecasting may be underutilized by AI policy organizations. But how much is forecasting actually being used? Which organizations working on AI—particularly AI safety—already integrate forecasting into their decision-making, policy recommendations, and strategic planning? What forecasting work is being done, and...
I envision a future more chaotic than portrayed in AI 2027. My scenario, the Rogue Replication Timeline (RRT), branches off mid-2026. If you haven’t already read the AI 2027 scenario, I recommend doing so before continuing. You can read the full version on my blog, but I have copied the...