I know a lot of the time people define intelligence like that because the AI field defines it like that[1], but when communicating with outsiders, I suspect you shouldn’t start the conversation by trying to change how they use the handles to understand the world.
The gap in definitions
For the average person, intelligence is primarily understood as the capacity to understand and learn new information easily.
We can link what I think are the two most common usages of the term:
Intelligence(AIField) = Intelligence(common) + Useful information + Goal-directedness + Agency
As you can see, the everyday usage and the technical definition used within AI safety are substantially far apart.
Why this matters for communications
If we want to help their maps of the world, I think there are better alternatives than messing with the handles they put on concepts. It must feel a bit fishy that someone who is trying to make a world-shattering claim is basing all of it on extrapolating from a term that they are using differently than you, Spock is your idea of smart, and you haven't ever thought about the achieving-general-goals property in much detail before.
But I've heard multiple people doing this in the AI Safety communications space.
Better framing alternatives
Instead of insisting that "intelligence IS the ability to achieve goals," we can use clearer formulations that convey the intended message without semantic friction:
“AI labs are trying to grow a digital species smarter than humans”
“AI companies are attempting to build highly knowledgeable agents capable of accomplishing goals across domains better than humans”
“In the AI industry, 'intelligence' means goal achievement; so when they talk about making AIs 'smarter,' they mean…”
One might argue that insisting on “intelligence is goal achievement” creates a simpler shorthand for associating advanced AI with danger, but I doubt it's better.
I think this is what Dwarkesh was getting at when he mulled over the difference between power and intelligence. You can be relatively dumb and still have power (the capacity to steer the future and achieve goals). The distinction is that if you control for power, say by putting two systems in the same exact conditions with the same exact goals, the more intelligent one will more reliably steer the future and achieve their goals.
I agree that equating intelligence naively with the capacity to achieve goals is an oversimplification, and not a good one for public communication.
I know a lot of the time people define intelligence like that because the AI field defines it like that[1], but when communicating with outsiders, I suspect you shouldn’t start the conversation by trying to change how they use the handles to understand the world.
The gap in definitions
For the average person, intelligence is primarily understood as the capacity to understand and learn new information easily.
We can link what I think are the two most common usages of the term:
Intelligence(AIField) = Intelligence(common) + Useful information + Goal-directedness + Agency
As you can see, the everyday usage and the technical definition used within AI safety are substantially far apart.
Why this matters for communications
If we want to help their maps of the world, I think there are better alternatives than messing with the handles they put on concepts. It must feel a bit fishy that someone who is trying to make a world-shattering claim is basing all of it on extrapolating from a term that they are using differently than you, Spock is your idea of smart, and you haven't ever thought about the achieving-general-goals property in much detail before.
But I've heard multiple people doing this in the AI Safety communications space.
Better framing alternatives
Instead of insisting that "intelligence IS the ability to achieve goals," we can use clearer formulations that convey the intended message without semantic friction:
One might argue that insisting on “intelligence is goal achievement” creates a simpler shorthand for associating advanced AI with danger, but I doubt it's better.
https://www.lesswrong.com/w/general-intelligence?lens=lwwiki-general-intelligence#Definitions_of_General_Intelligence