One advantage of P(doom) being unspecific is that the probabilities people have for it are also unspecific and are generally guesses rather than carefully calculated numbers. Having more specific types of catastrophe probabilities depending on the scenarios could portray false specificity.
I think there's a few reasons why RSI doesn't seem to be priced in. Here are some different RSI -> ASI scenarios:
This congressional letter does a decent job at synthesizing the main logic: https://democrats-foreignaffairs.house.gov/wp-content/uploads/2026/11/letter-to-commerce-on-ai-chip-export-to-chinafinal.pdf . The main evidence is an NYT article based on anonymous sourcing about US intelligence which of course isn't the strongest. However, I do think that generally it would be challenging to have good public evidence that someone is a spy because the point of spying is to be hidden.
I would contest the assumption that safety & monitorability tech don't impact the chances of a pause. I think it's possible that safety work increases pause chances because it makes the danger of ASI (plus AI systems on the path to ASI) more visible. Many of the scariest components of misalignment events come to light via monitoring or interp.
Super interesting! I definitely agree that intrinsic motivation for alignment would be preferred. However, I sort of wonder if permanent monitoring like what the major labs are doing now would be treated as extrinsic motivation? Maybe this monitoring does result in a permanent-ish state of "with the line"?
I think another thing to highlight here is a loss of trust in Jensen's judgement. Prior to this incident, it was plausible to think that Jensen's interest in promoting chip sales to China was based on either benefiting NVDA shareholders or because he truly believed it would prevent China from developing competing AI hardware.
Now, there's an additional layer where his motivations in chip sales to China could have been influenced by his relationship with Alice Huang rather than what's best for the US or NVDA shareholders.
It also seems like Alice Huang was re... (read more)
Interesting post! One thing I think is good is the ability to keep existing models running which does seem to lower incentives for hiding chips and makes implementation more practical.
An issue with this plan though is that it could prompt lagging labs to fight against it. This is because it makes the current model intelligence hierarchy concrete which advantages current frontier companies over labs that believe they could catch up. This could lead to large lobbying efforts against the plan from Meta, XAI, Nvidia, Chinese labs, etc. I think it also could gi... (read more)
Another possibility is terrorists or other bad actors use AI to make a bioweapon that causes serious but non-x-risk damage.
Even though others could use AI to make a vaccine or a cure, the public reputation of AI companies vastly decreases and there are calls to shut down all development.
I could see this sort of thing happening with a jailbroken/misaligned closed-weights model or an open-weights model.
Not OP and haven't run this experiment on myself, but I think this post is pretty useful on the general strategy https://www.lesswrong.com/posts/eaxoBCmQd4vew9ZXX/pseudpocalypse