Top postsTop post
Cross-posted from my website. I continue to believe we should pause frontier AI development. Any discussion of alternative strategies should be thought of as planning for contingencies. "Super-persuasive" AI is dangerous because a misaligned ASI could persuade humans to help it take over. But setting that aside, even if we...
An argument goes: If we don't build aligned artificial superintelligence, we risk driving ourselves extinct for some other reason. We should rush to build ASI quickly, in spite of the risks—the longer we wait, the more vulnerable we are to extinction from a different cause. Other than ASI, the biggest...
Cross-posted from my website. Prior discussion: niplav's shortform (2025); Planning for Extreme AI Risks (2025) by Joshua Clymer A frontier AI company (any one, I don't care which) should close shop and make an announcement along the lines of: > Powerful AI could end the human race. We are too...
To prevent superintelligent AI from killing everyone, I would like there to be a strong international agreement banning the development of ASI until it can be proven safe. But that sort of agreement requires a lot of political buy-in and coordination. In the meantime, it may be easier to get...
Cross-posted from my website. Depth-first plans lay out a path from here to aligned superintelligent AI. We need those kinds of plans. But depth-first plans depend on many assumptions: "We will make AI safe by doing step 1, then step 2, then step 3." Step 1 only works under condition...
Cross-posted from my website. Three categories of futures, depending on how AI goes: 1. ASI timelines are long. 2. ASI timelines are short, and we're on track to solving AI alignment. 3. ASI timelines are short, and we're not on track to solving AI alignment. If we want to make...
Cross-posted from my website. If civilization advances its technological capabilities without advancing its wisdom, we may miss out on most of the potential of the long-term future. Unfortunately, it's likely that that ASI will have a comparative disadvantage at philosophical problems. You could approximately define philosophy as "the set of...