Alek Westover's Shortform

Alek Westover

This is a special post for quick takes by Alek Westover. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

I worry that AI companies focus too much on arguing about risks from their current models compared to the amount of time they spend discussing how much catastrophic risk they plan to impose in the next year. This is bad because it's very hard to mitigate risk at the last second.

AI companies should publicly state how much risk they intend to impose in the next year and defend their claim to the public. Regulators should require this practice. If AI companies don't have credible public plans for how they expect to keep catastrophic risk below 2% per year, they should inform the government and the public of this fact, and recommend that the government shut them down and work towards internationally shutting down efforts to make more powerful AI models.

I like a situation where AI companies are told to publish the entire evidence base about future risk. like everything a reasonable outsider would need to make the estimate. And then the conversation happens outside. I’m worried if we told ai companies to give p dooms they would just lie. It’s too easy to say any number, because we can’t audit the epistemics.

1rcc3h

Both Anthropic and OpenAI are already pretty open about the fact that their research poses significant catastrophic risks. I don't think forcing them to officially assign a number to it will directly affect mitigations - any quantification here is guesswork. It may have some effect on the public opinion though but that can be achieved in a Senate hearing, doesn't need actual regulation.

2Hastings4h

Forecasting is hard! perhaps instead of a prediction, just a bound? , but if quantifiably upper bounding the risk is beyond their capabilities, well, that also is important to know.