The reason I wanted to make this linkpost now rather than some other time is because discussions over AGI and whether or not LLMs are or aren't AGI are happening right now, and the point of the linkpost is that the term AGI is for our purposes useless at this point, because we are now in the fuzzy cloud now that AI can do real economic work.

Some choice paragraphs:

It used to seem possible that, in practice, the differences between these definitions might not matter all that much. If AI capabilities progressed smoothly—say, from mouse- to chimp- to human-level, or from preschooler to high schooler to PhD graduate—then all of the above definitions might be fulfilled at roughly the same time.

These days, though, it’s clear that AI capabilities are highly jagged. This makes it likely that there could be big gaps between when different definitions of AGI are fulfilled.

I think this was an underrated reason for why the term AGI lost its value, because as it turned out, certain definitions that do connect/correlate very well in humans are easier to disentangle than we thought for AIs (though humans are also jagged, but we don't realize because we are comparing against our own baselines).

On the usefulness of AGI 1-2 decades ago:

It’s not inherently bad for a concept to be fuzzy - many useful concepts are. Love, life, justice, freedom… As long as invoking the concept lets you gesture in a direction that your listener understands, fuzziness doesn’t have to be a big problem.

For a long time, this was the situation with AGI: we were far enough away from the fuzzy cloud that the specifics didn’t really matter. Back when the term was coined in the early 2000s, the point of talking about artificial “general” intelligence was to contrast with the “narrow” AI systems that existed at the time. “Narrow AI” refers to models that can each do one specific thing, like recognizing zip codes, detecting credit card fraud, or filtering spam. Back when the AI that we could build in reality was far from any definition of AGI, it was helpful to be able to gesture in the direction of much more capable, general-purpose systems that we might one day develop:

Now I should stop here to note that the gesturing was likely wrong on overestimating how good AIs would become once they got good at language, and there are a couple of reasons for this, but the big ones are that AIs could be capable without being nearly as sample-efficient as humans, either at pre-training or post-training, and this probably made them assume less jaggedness in AI than currently exists, and the other big one is assuming more neuralese/recurrence for AI than what currently exists, and as it turned out the direction AI would go in would deemphasize forward passes in favor of Chain-of-Thought, which improved capabilities and interpretability (this is demonstrated by the fact that No-CoT doubling times are a little over a year, compared to general doubling times on the order of 100 days), but critically only boosted AI capabilities in an interpretable manner.

Now onto this:

But that’s changed. Today’s best AI systems are good enough that they’re now inside the fuzzy conceptual cloud of “AGI-ish”: that is, they’ve surpassed some people’s definitions of AGI, while falling well short of others’. As a result, talking about “AGI” is no longer a helpful way to gesture in a rough direction—instead, it’s likely to make some people think you mean one thing, and others imagine something totally different:

Yep, I think that once AIs could do real economic work, which I'd peg at November 2025, the term AGI lost all of it's value, and the question is what more specific terms are necessary in order to recapture the lost value.

Here's one use-case for more specific terms

In my experience, there are two big use cases for a term like AGI:

1. Predicting a phase change: “Sure, things are a certain way now, but once we have AGI, things will be a whole other way…”
2. Predicting we’re not close to the ceiling of AI progress: “AI has gotten a lot better over the past few years, but there’s still a long way to go…”

If you want to talk about a phase change, I think the best approach is to be as specific as you can about what milestone you think will trigger the phase change and why. Some options:

• Fully automated AI R&D (which could lead to an intelligence explosion)
• AI that is as adaptable as humans (which could lead to massive, irrecoverable job loss)
• Self-sufficient AI (which would no longer need humans for energy, chips, or anything else, and could therefore dispense with them)
• AI becoming conscious or otherwise worthy of moral status (which should radically change how we interact with AI)

Also valuable is that with more specific terms, you open yourself to a greater risk that your theory is falsified, which is good, but not how humans normally reason, unfortunately.

Another use-case that needs more specific terms:

If you want to talk about a high ceiling, one good option is just to describe that: “AI that’s far more capable than what we have today.” If you want a single term, the simplest option is “superintelligence,” which roughly means AI that is far smarter than humans in most or all ways. Like AGI, superintelligence is a fuzzy cloud of a concept, but it’s still far enough away from the AI we have today that it can be a helpful direction to gesture in, just like AGI was 20 years ago.

(This is a good time to note that I have a disagreement with Helen Toner that I also don't think superintelligence is that far away in some domains because of inference scaling, and while inference scaling has it's limits because it requires verifiability that only exists for a relatively small portion of the current economy, in the domains where this does exist, it's conceptually easy to scale current systems to superintelligence in those domains).

And finally, the reason the linkpost exists at all:

The gaps between different conceptions of what “AGI” even means are starting to damage our ability to think together about how to navigate the transition to a world with extremely advanced AI. A word that lets three people think they’re talking about the same thing—when one of them means “o3 with tool use” another means “AI that could run civilization autonomously without humans,” and the third means “an AI that works exactly like the human brain, consciousness and all”—is an actively anti-helpful word.

This is a huge problem, but one that is kind of being worked on by Daniel Kokotajlo and others at the AI Futures Project, which used more specific terms and moved away from the term AGI in it's modelling (which I thank them for doing).

And yeah, AGI/superintelligence has at this point become far too vague, unfortunately to make it a useful term. It was useful 1-2 decades ago, but we should aim to stop relying on it now that we have more evidence.