Ian McKenzie's Shortform

Ian McKenzie

This is a special post for quick takes by Ian McKenzie. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

There are two narratives of the Hugging Face incident that I keep seeing that and that keep bothering me:

This is a marketing ploy by OpenAI to somehow make themselves look good
Talking about the models being misaligned is 'anthropomorphizing' and lets OpenAI off the hook.

Take (1) is so confused that I don't even really want to engage with it here, but I'm worried that take (2) is going to be a mainstream take (e.g. this Cal Newport NYT article, where he says "if Ford automobiles were spontaneously catching on fire, we wouldn't tolerate the chief executive's changing the subject to discuss our general lack of preparedness for a future with flying cars. We'd demand that the company address the acute problem happening right now.") and that it will create bad incentives, even among people who dislike OpenAI.

To spell out the obvious: it's clearly OpenAI's fault that the Hugging Face hack happened. This would be true even if the agents were literally conscious or 'sentient' or whatever – OpenAI created them. OpenAI is responsible for the things it creates. But a focus on fixing 'the acute problem' directs effort towards the visible parts of the problem while putting no pressure on the company to actually prepare for the future. To torture Newport's analogy, maybe Ford could get away with putting fire suppression systems in their cars to prevent the current round of fires, and this would appear good enough to consumers. But if we then we keep putting bigger and bigger power sources in the cars and we don't understand why they sometimes catch fire, then we're on track for a massive explosion.

None of this is new to a LessWrong audience but I guess I'm just sad that even when we have a massive warning shot, the anti-OpenAI sentiment that is generated is still somehow completely misdirected.

4silentbob2mo

My best guess for the nature of #2 is that some people like Cal Newport or Internet of Bugs (and to a degree Ed Zitron) have a strong tendency to moralize issues, seeing the world through a lense of human intent being the main driver of everything that happens, and hence you constantly need to focus on identifying culprits and holding them accountable, or at least convincing your audience to condemn them. If human intent is what drives everything, then talking about the technology itself becomes problematic, as it distracts from the thing that actually matters, which is judging the bad people. In a response to Anthropic's post on recursive self-improvement, Newport said "I would almost say shame on Anthropic for dropping this report. No suggestions, no reassurance, no culpability, no responsibility." Newport is also particularly drawn to explaining some aspects of how LLMs work (with varying degrees of correctness), to then conclude that clearly they cannot have goals or intentions. When your main goal is to find the bad people and make everyone agree with you that they are bad, then you can neither grant any agency to AI, nor would you accept the more systemic issues or coordination problems to play an important role.

1Haiku2mo

I think you can get people unstuck from that prior if you offer them a line of retreat toward an even bigger reason to hate the AI companies. My current go-to snappy retort for use in low-quality discourse (e.g. YouTube comments) is: "Exactly. Just like how oil companies invented climate change to make oil sound cool to investors." (I don't have any evidence that this works, but I predict that little nudges and intuition pumps like this would work over time if enough people offer them.)

I kind of doubt that people would be nudged away from their "doom is just marketing" lense by snappy reports. But maybe the question is more what others who then read the exchange take away from it, as many of them may not have any strong opinions yet.

I would guess that from the overall population, some small percentage is following the "doom = hype" narrative, and some probably smaller percentage is following the "AI is a dangerous technology and any incidents that happen show that we're getting closer towards existential risks" narrative, with the vast m... (read more)