This is a special post for quick takes by Linch. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
In technical conversations where people treat the "normal technology"/"normal engineering" thesis seriously, make it as clear as possible that AI companies are obviously not taking all the lessons from "normal technology," and obviously less safe than other engineering disciplines.
Right now some of the debate is between people who treat AI as normal technology and therefore we can handle it like normal technology (subtext: this means without much problems) vs people here who think obviously it's not a normal technology.
I think we're mostly winning the "not normal tech" argument in the public and the media, but I also think that even if you take the "normal technology" thesis seriously, their conclusions don't follow.
In particular, they elide past how normal technology has historically been made safe after hard-won lessons from multiple incidents and piles of corpses. Normal technology people never mention the body count (optimistically because they did not read much of the history of normal technology).
I think we mostly won the debate against people who say there's no risks, at least within AI companies (so we won the debate against capabilities skeptics, e/accs who refuse to acknowledge costs and just think we should let 'er rip, and Pollyannish types who think there aren't risks even after superhuman AI)
Right now our opponents (especially within AI companies) are mostly people who agree that this tech poses an extraordinary risk, but still think that "normal safety practices" suffice to make things safe, based on the history of normal engineering. [implicit: those lessons are applied, and also we can just do the first-order trial-and-error thing and it'd be fine, despite these risks being at very different scales]
I'd like to continue to shift the conversation for the informed range to be something like pessimists say that humanity hasn't learned enough to make ASI go safely without a lot more careful theory and analysis, and therefore we need pause/slowdown, and optimists to say something more like "this technology poses an extraordinary risk, however the history of ordinary engineering and other fields of human endeavor are sufficient for us to guard against the risks of superhuman intelligence, and here are all the ways we're actually learning from other fields of human endeavor on how to make it safe the first time."
In policy, I want to shift the conversation to force safety opponents to clearly lay out the case that the benefits are higher than the costs. I expect this to have multiple benefits:
I think the conversation will be healthier overall, and also maybe there are specific keyhole approaches for people to get some of the benefits without incurring insane costs.
I expect us to win the debate on those terms, especially on the current margin.
Once it's normalized, opponents who deny this framework would be more clearly seen to be dodging the question.
Man, I agree that there needs to be some ability to talk about adversarial dynamics in the discourse, and public messaging conflict, but I just feel so much doom when I hear people refer to this kind of adversarial attitude towards comms.
I am not quite sure what I think people should do instead. Of course at some point you need to take these dynamics as a given and move on assuming that "your opponents" will never change their mind, but I guess I want there to be some moderation on approaches that do that.
Not sure what this means for this comment, but I figured I would express my reaction. I don't have a strong take on what the norms should be, at least right now.
Nate keeps pointing out in podcasts that it's like: "there's hope because it's not that the bus driver is gladly driving off a cliff, it's more like the bus driver is asleep at the wheel and we just have to wake him up" seems like the correct attitude to have to me.