I was very surprised today on a podcast to hear Jensen Huang plainly state that if they cannot align the AIs, then the labs must shut down.
The context I have on Huang is that he has run NVIDIA for 30+ years, which has become the most valuable company in the world due to the AI boom. My understanding is that he has repeatedly encouraged the US President (with whom he is on friendly terms) to continue to support AI, and dismissed AI talk as "sci-fi".
If you haven't seen, his biographer has incredible quotes of him being pressed on risks from AI, where Jensen gets furious.
“This cannot be a ridiculous sci-fi story,” he said. He gestured to his frozen PR reps at the end of the table. “Do you guys understand? I didn’t grow up on a bunch of sci-fi stories, and this is not a sci-fi movie. These are serious people doing serious work!” he said. “This is not a freaking joke! This is not a repeat of Arthur C. Clarke. I didn’t read his fucking books. I don’t care about those books! It’s not– we’re not a sci-fi repeat! This company is not a manifestation of Star Trek! We are not doing those things! We are serious people, doing serious work. And – it’s just a serious company, andI’m a serious person, just doing serious work.”
And interviewed by Dwarkesh Patel he says other dismissive things:
“If we scare this country into thinking that AI is somehow a nuclear bomb, so that everybody hates AI and everybody’s afraid of AI, I don’t know how you’re helping the United States. You’re doing it a disservice.”
Now, on today's NYT podcast, here is my transcription of the relevant excerpt, coming off a discussion of the Hugging Face attack. Relevantly, Hugging Face is a company that Jensen Huang has recently acquired!
Ezra Klein:. ..and in a way that was capable of causing tremendous damage. And so on the answer of "you just have to align them"... I guess what I'm hearing from people at these labs is like, they're not sure how to align them.
Jensen Huang: Well in that case they shouldn't release the product. That's the simple answer. If you're gonna build a self-driving car, let's say it's a robo-taxi, and there's a really difficult condition, and as an engineer we say we have no idea how to solve this problem... because these cars are not programmed, they're trained... and so we have no idea how to train these cars, and we have no idea how to align them to the safety standards that are expected on the road. And so what's the answer is: don't ship it.
Ezra Klein: These products weren't released.
Jensen Huang: What's that?
Ezra Klein: These products weren't released.
Jensen Huang: Ah, now it comes back to engineering problem again. And so, one, you have to root cause it, what you could have done, what's the solution for it. And then in the future you just improve your process so that you could avoid this from happening again.
I am fairly certain they will say, yes, they know how to solve this problem. And if that's the case, then that's the problem. It's as simple as engineering.
And now the alternative, is that, if they say that there is no way to contain our experiments... there's just no way. If we test our AI models, it will get out, and it will damage the world. Then I think the answer is we have to shut the labs down.
Because the cost to humanity... the damage is too great. It could be civil liabilities, it could be criminal liabilities. The liabilities are incredible.
Ezra Klein: If they hacked you while Hugging Face was your product, would you sue them? Or press charges?
Jensen Huang: It depends, of course. Obviously, if damage was done to our company, we would have to take, we have to consider all options. There's so many laws. There's cyber laws, there's product liability laws, there's all kind of laws. Damaging property laws. There's all kinds of laws.
And there you have it folks! If they cannot be aligned, one of the top accelerationists thinks, if they cannot align their AIs, they should be shut down. This is really only one or two arguments away from the position of wanting to regulate them, and also from shutting it all down. From my perspective is a step forward the public conversation.
My gut reaction is that this is just a rhetorical move he and others (e.g. Sacks) landed on that seems to hit well with some people, but they don’t actually believe it.[1]
They reject the idea of losing control, and Jensen downplays AI as being same as standard software we can just engineer.
I think it’s more like, “it’s fine for losers who believe in AI risks to shut down, while actual real engineers will solve problems like we always have. I’m happy to take the reins if these fearmongers want to stop.”
He doesn’t believe in AI risk and would be happy to keep building (or empowering others who repeat the party line) if other companies believe it strongly enough to stop or slow down.
In the podcast, it also seems he failed to recognize that his argument does extend to what actually happened in the HuggingFace example since he says they should just “not release the product.”
They are likely just trying to “call their bluff” by syncing their statement with the reasonable view of “if you believe it would kill us so much, why aren’t you stopping?” They don’t appreciate the race dynamics, consider alignment to be harder than normal software, and want to land on a take that effectively points towards no regulation.
there is some sense in which i wonder is jensen is just deeply irritated and/or feels it is very unfair that the world can be so inconvenient such that weird scifi shit like the singularity can mess with him running his company. i think he wants to just run his company, make $$$, and be at peace. alas, reality is...disinclined to acquiesce
I was very surprised today on a podcast to hear Jensen Huang plainly state that if they cannot align the AIs, then the labs must shut down.
The context I have on Huang is that he has run NVIDIA for 30+ years, which has become the most valuable company in the world due to the AI boom. My understanding is that he has repeatedly encouraged the US President (with whom he is on friendly terms) to continue to support AI, and dismissed AI talk as "sci-fi".
If you haven't seen, his biographer has incredible quotes of him being pressed on risks from AI, where Jensen gets furious.
And interviewed by Dwarkesh Patel he says other dismissive things:
Now, on today's NYT podcast, here is my transcription of the relevant excerpt, coming off a discussion of the Hugging Face attack. Relevantly, Hugging Face is a company that Jensen Huang has recently acquired!
And there you have it folks! If they cannot be aligned, one of the top accelerationists thinks, if they cannot align their AIs, they should be shut down. This is really only one or two arguments away from the position of wanting to regulate them, and also from shutting it all down. From my perspective is a step forward the public conversation.