Book review: The Thinking Machine: Jensen Huang, Nvidia, and the
World's Most Coveted Microchip, by Stephen Witt.
This is a well-written book about the rise of deep learning, and the man
who is the most responsible for building the hardware that it needs.
Building the Foundations
Nvidia was founded in 1993 by three engineers. They failed to articulate
much of a business plan, but got funding anyway due to the reputation
that Jensen had developed while working for LSI Logic.
For 20 years, Nvidia was somewhat successful, but was an erratic
performer in a small industry.
Around 2004, Nvidia increased the resources it devoted to parallel
processing. There was probably some foresight involved in this strategy,
but for around a decade Nvidia's results created doubts about its
wisdom. The market for such GPUs was tiny. Most experts believed it was
too hard to make parallel software work.
But Jensen excels at solving problems that most others would reject as
impossible.
The Most Important Decision
Nvidia was sometimes less interested in neural networks than neural
network researchers were interested in Nvidia. Geoffrey Hinton sometimes
couldn't get Nvidia to respond to his emails (circa 2009?).
In 2013, Bryan Catanzaro proposed that Nvidia support work on
cuDNN (software to run deep learning
on GPUs). The reaction from Nvidia's software team was negative.
Catanzaro appealed to Jensen. Jensen quickly developed something like an
expert understanding of deep learning, and soon announced that AI was a
"once in a lifetime opportunity". He made AI Nvidia's top priority.
It took a fair amount of skill to be positioned to profit from AI. But a
large fraction of Nvidia's success depended on that one decision. I
estimate it was worth a trillion dollars.
Jensen ascribes Nvidia's success to "luck, founded by vision".
Jensen's Character
What drives Jensen? The book provides no clear answer. It doesn't seem
to be money. I'm guessing it's some measure of business success along
the lines of market share.
There are some obvious similarities to Elon Musk. A few key differences:
Loyalty: Jensen almost never fires employees.
Musk's visions involve backchaining from fantasy; Jensen looks
forward from reality.
Jensen has a stable marriage; Musk has something else.
Jensen is pretty cautious about expressing political opinions[*];
Musk's businesses have been hurt because he's vocal about
politics.
Witt claims that Jensen "never offered a single political
opinion", but I found opinions on immigration policy and export
restrictions.
Jensen often yells at employees. But that always seems calculated to be
productive.
He appears to have above average integrity. The only act described in
the book that I'm inclined to classify as unethical is filing a
possibly frivolous lawsuit against competitor 3dfx, at a time when the
legal fees were enough to put 3dfx out of business.
I should note that after the book was published, a report was
published
about Nvidia exerting questionable pressure on the news media.
AI Risks
Another difference is that Jensen insists there are no risks from AI
which are worth our attention. Witt is quite concerned ("It seemed to
me that the end of my profession was approaching. It seemed to me that
the end of reality was approaching. It seemed to me that the end of
consciousness was approaching."), and annoyed Jensen by pushing hard
on this topic.
Jensen isn't willing to say enough on this topic for us to infer much
about his reasons. I'll conjecture that he's mostly reasoning from
what's good for his business.
I expect that if a political situation causes Nvidia's business to be
impaired by alignment concerns, Jensen would devote some unusual thought
to handling alignment. I wish I could tell whether that meant using his
intelligence to actually solve alignment, or to convince the world not
to worry about alignment.
I see a ray of hope from a 2023 interview, in which he said: "No AI
should be able to learn without a human in the loop.". It won't be
long before he'll see that this is a real issue. Maybe enforcing such a
rule will be enough to keep us safe. Or maybe he'll see that the rule
can't be enforced, and he'll switch to advocating a more enforceable
rule.
Book review: The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip, by Stephen Witt.
This is a well-written book about the rise of deep learning, and the man who is the most responsible for building the hardware that it needs.
Building the Foundations
Nvidia was founded in 1993 by three engineers. They failed to articulate much of a business plan, but got funding anyway due to the reputation that Jensen had developed while working for LSI Logic.
For 20 years, Nvidia was somewhat successful, but was an erratic performer in a small industry.
Around 2004, Nvidia increased the resources it devoted to parallel processing. There was probably some foresight involved in this strategy, but for around a decade Nvidia's results created doubts about its wisdom. The market for such GPUs was tiny. Most experts believed it was too hard to make parallel software work.
But Jensen excels at solving problems that most others would reject as impossible.
The Most Important Decision
Nvidia was sometimes less interested in neural networks than neural network researchers were interested in Nvidia. Geoffrey Hinton sometimes couldn't get Nvidia to respond to his emails (circa 2009?).
In 2013, Bryan Catanzaro proposed that Nvidia support work on cuDNN (software to run deep learning on GPUs). The reaction from Nvidia's software team was negative. Catanzaro appealed to Jensen. Jensen quickly developed something like an expert understanding of deep learning, and soon announced that AI was a "once in a lifetime opportunity". He made AI Nvidia's top priority.
It took a fair amount of skill to be positioned to profit from AI. But a large fraction of Nvidia's success depended on that one decision. I estimate it was worth a trillion dollars.
Jensen ascribes Nvidia's success to "luck, founded by vision".
Jensen's Character
What drives Jensen? The book provides no clear answer. It doesn't seem to be money. I'm guessing it's some measure of business success along the lines of market share.
There are some obvious similarities to Elon Musk. A few key differences:
Jensen often yells at employees. But that always seems calculated to be productive.
He appears to have above average integrity. The only act described in the book that I'm inclined to classify as unethical is filing a possibly frivolous lawsuit against competitor 3dfx, at a time when the legal fees were enough to put 3dfx out of business.
I should note that after the book was published, a report was published about Nvidia exerting questionable pressure on the news media.
AI Risks
Another difference is that Jensen insists there are no risks from AI which are worth our attention. Witt is quite concerned ("It seemed to me that the end of my profession was approaching. It seemed to me that the end of reality was approaching. It seemed to me that the end of consciousness was approaching."), and annoyed Jensen by pushing hard on this topic.
Jensen isn't willing to say enough on this topic for us to infer much about his reasons. I'll conjecture that he's mostly reasoning from what's good for his business.
I expect that if a political situation causes Nvidia's business to be impaired by alignment concerns, Jensen would devote some unusual thought to handling alignment. I wish I could tell whether that meant using his intelligence to actually solve alignment, or to convince the world not to worry about alignment.
I see a ray of hope from a 2023 interview, in which he said: "No AI should be able to learn without a human in the loop.". It won't be long before he'll see that this is a real issue. Maybe enforcing such a rule will be enough to keep us safe. Or maybe he'll see that the rule can't be enforced, and he'll switch to advocating a more enforceable rule.