This is a special post for quick takes by Lukas_Gloor. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
[Are security researchers becoming polarized against AI risk concerns?]
I noticed a bunch of accounts on X with "security" in their bios saying stuff like AI risks are overblown and the real problem is companies have bad security, etc. These accounts tend to be very dismissive of AI risks while saying sensible things about security. It might be a coordinated campaign and not representative of the security community as a whole, or it might be that AI risk discourse has struck a particular nerve and now there's a risk that security researchers form an identity of being anti AI risk concerns?
I don't know what's going on, but I'd like to draw attention to the possible trend, and I'd feel more reassured if someone with a lot of credentials in security would say something like "both are true, companies suck at security but also AI risk would be worth taking seriously even if all companies everywhere in the world suddenly stopped sucking at security."
I'm not sure I'd jump to it being evidence of an explicit conspiracy myself. Part of it seems like Girardian mimesis in action. Part of it seems like good old fashioned marketing - "Unlike those bums who sit around UC Berkeley all day and get weird when a friend orders shrimp scampi instead of fried tofu for dinner, I care about real problems."
I work in cybersecurity myself, a fact which continually amuses me. I can attest that it's tough in the best of times to get companies to actually pay for security until something goes wrong. Nobody ever gets credit for fixing problems that never happen, after all. Anthropic reasoning suggests that AI safety in the sense of "an unaligned ASI kills everyone on Earth" is almost definitionally a problem that never happens to anyone who sticks around to give or get credit for it. Bad news if you need to make rent! So I suspect this kind of trend would show up organically in most worlds similar to our own.
This is the "we can just contain the SI" argument. I agree with it as a short-term thing to do and if you engage you need to wind up in a both/and for that reason. "We should try harder to secure it, but we don't want to wind up in a situation where one mistake gets a frontier-level agent worm loose on the internet." Something like that?