Don't call it a "pause", as that messages that a pause is much weirder than it is

less_raichu

Eric Schmidt on CNN: "we're not going to pause AI progress because the entire incentive system in America is not to pause." And he added, it's hard to enforce a ban across all labs.

The part about inter-lab enforcement is important, but that's not the only thing he said. He also said, labs lucid about x-risk are incentivized to not pause. This is a rather bald-faced lie, as corporations are allowed to not do things that might destroy the corporation.

If p(doom) = 100% and it's common information (everyone knows everyone knows, ad infinitum), we simply use the word "can't", and we don't think twice about it. Someone still could, but that person is considered crazy, and our regulatory question is how to stop them.

I'm handwaving a lot of game theory here^[1], but I don't crux on it when I say "don't call it a pause". In terms of the public debate, I'd like Eric Schmidt to assume burden of proof that the labs are required to do the dumb thing. I don't think he should get to hide behind the word "pause" when dismissing a pause as nonsense.

More shared knowledge to not do the dumb thing, is good, because we don't pay an overhead from mismodeling each other's incentives, and we can't disturb a "don't build it" equilibrium as easily.

Talking about a pause dilutes that shared knowledge. As long as the very concept of "pausing" is in our lexicon, rushing to build unsafe ASI is the default.

^{^}
They all have different p(doom), which is an overly coarse concept, and additional x-risk for being the 2nd lab to act is a factor, and common information is shaky, and I don't know how to model what investors want. Plus, I don't know how to model, why did Eric Schmidt benefit from framing it like that, if he knows about x-risk in the first place.

I'd like Eric Schmidt to assume burden of proof that the labs are required to do the dumb thing [...] More shared knowledge to not do the dumb thing, is good

What is "the dumb thing", specifically? I'm imagining that training or releasing GPT-6 Astra or Claude Fable 5.1 wasn't "the dumb thing", because those are fantastically valuable commercial products that have created enormous amounts of economic value (far in excess of the damage caused by the reported hacking incidents in evaluations). Or was it? Maybe GPT-4 was the dumb thing, and everything after that was even dumber? What is it?

To be clear, I personally agree that the near-term extinction risk from advanced AI is terrifying, and that a pause or slowdown would be good. But in order to appeal to "shared knowledge" that a pause or slowdown is the default, non-insane action (such that "pause" language is misleadingly marking what should be unmarked), you should be able to answer this kind of specific, nitpicky question. Otherwise, you don't have "shared knowledge" or "common information"; you're just pushing your preferred frame like any other political actor. Maybe you think that's the winning move, but I think we might still have need for shared knowledge of the actual meaning of shared knowledge, even at this late hour.

you're just pushing your preferred frame like any other political actor

Yeah, that's what I meant by "messages," but fair enough, I should defend if it's better epistemics or just a trick that won't stick. So yeah I do think it's better.

Option A: Lab CEOs commit to a first-ever pause/slowdown/pacing, Schmidt and others dismiss it as nonsense violating basic capitalism sense, it's voluntary, it leads to charges of collusion, and they're actually getting sued for collusion.
Option B: We can't make them any bigger without serious risk, we don't know how, so we're not. Lab CEOs meet to solve alignment to speed up development again, which is not an antitrust violation.

Option A invents a new concept for do something unremarkable, and puts burden of proof on whoever says "pace", even though pacing is just a new word for something unremarkable.

Sam Altman did make a big concession toward Option B on 9/16: ( https://openai.com/index/responding-next-frontier-critical-cyber-capabilities/)

We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.

I think I can answer that the dumb thing is "scaling at maximum speed for much longer." It's still coarse and more shared knowledge about where that line is would be great, but it's good enough that I'd like to hold Sam accountable to it.