Epistemic status: Fairly clear evidence, and seems useful to understand.

What is the most informative way to measure intelligence?

In other words, how hard is it to make as ASI that could reasonably be described as having IQ 300? How about IQ 3000?

In theory, any pair of ways of measuring intelligence that are related by a monotonically increasing function are both valid. However, that doesn’t mean that one of them isn’t more informative or intuitive than the other.

We are by now rather used to seeing AI improve and pretty quickly saturate one eval after another:

Any individual eval or task shows a logistic curve over time, as AI’s chance of doing it improves from <10% to >90%.^[1] Different evals saturate at different speeds.^[2] Many evals are deliberately designed so as to have a mix of easy tasks, intermediate tasks, and difficult tasks so that they take longer to saturate. If you don’t deliberately do this, typical evals tend to saturate in somewhere between 6 and 24 months: say about a year on average.

This graph is using date as the x-axis. Over the last few years, AI training run compute has been increasing at – per year. Due to algorithmic (and training data filtering) improvements (but setting aside the effect of a couple of dramatic early improvements, like the invention of the transformer, that changed the scaling law slope, whose effects thus compound over time), the effectiveness of compute has been increasing perhaps – a year,^[3] for a total increase in effective compute of about an order of magnitude per year. So rather than labeling the x-axis as “Date”, we could equally well label it as “” (where compute is shorthand for effective compute), and replace the year ticks on it with order-of-magnitude ticks.

Most evals now saturate in between a half and two orders-of-magnitude in effective training compute (about 1 OOM on average).

Note that if you instead used a linear scale in effective compute, then the eval saturation curves would not be logistic curves, and would instead be distinctly asymmetrical. This would be a less useful (though still formally valid) way to graph the data. The elegance of having the success chance be a simple, symmetrical, logistic curve with a fairly consistent range of saturation rates privileges the logarithmic scale as a particularly useful way to measure intelligence: it’s the most natural scale to use.

Interestingly, the psychologists who specialize in measuring human intelligence have also studied something very similar. A number of different scales for measuring IQ have been defined. For example, the well-known Stanford-Binet one is based on grading on a curve: specifically a normal curve with median IQ 100 and standard deviation 15 IQ points (or before its 5th Edition, 16). Another common (and newer) type of IQ tests, Item Response Theory (IRT) ones, instead look at specific questions used on the IQ test and attempt to use a scale where the chance of a person answering that question correctly (or their chance above random chance for guessing a multiple choice question) is a logistic curve. Different questions saturate at different rates, generally with it taking anywhere from 25 IQ points to 70 or more IQ points for a specific question to saturate from 10% to 90% (say around 50 IQ points on average):

Interestingly, the Stanford-Binet and IRT-based measures of IQ match each other pretty-much linearly over roughly the IQ 70–IQ 145 range (above which we don’t have good statistical data), and the nonlinearity found below IQ 70 looks rather like the effects of a “fat tail” of major disabilities causing significant mental impairments and thus making the distribution genuinely not a normal distribution (which Stanford-Binet then forces back to a normal distribution by definition, while IRT does not) — just as we find for the distribution of most other human capabilities (such as the effect of dwarfism and similar issues adding a fat tail to the normal distribution of height). So that strongly suggests that the familiar human IQ scale is in fact approximately logarithmic in “equivalent effective training compute”: i.e. that adding something like 50 IQ points to an AI-simulated-human model (if we knew how to train an AI model whose skill profile matched human, rather than being very spiky in comparison) would require using about an order of magnitude more training compute for it. (Note that this observation is derived from data spanning a range of about 90 IQ points, more than enough for a range of questions of different difficulties to each go through most of their logistic curves — so we genuinely do have enough data here to distinguish linear from logarithmic.)

Admittedly, it seems rather implausible that if you compared two people with an intelligence difference of ~50 IQ points, the synapses in the smarter person’s brain could actually be generating about an entire order of magnitude more raw compute: the size and metabolic load of human brains don’t vary by anything like that much. So presumably there is some sort of algorithmic efficiency, quality of training, or quality of genetically-determined priors effect going on in humans that explains quite a lot of the differences in effectiveness/IQ between them, rather than actual large differences in raw synaptic compute. Or perhaps part of the explanation is that, as Moravec’s paradox demonstrates and as is well known in neuroanatomy, the large majority of the neurons and synapses in human brains are devoted to doing subconscious things like visual processing or muscular coordination that don’t show up on an IQ test, and IQ is actually measuring just how good a job a rather small fraction of all the neurons are doing: mostly just the ones that do conscious abstract System 2 thinking — a frction small enough that this fraction could plausibly vary quite a bit between people: some people might actually manage to have significantly more of their neurons contribute to it than others do.

As a rough model of improvements in AI over the last 5 years, AI adding something in the region of 50 IQ points a year doesn’t sound that far off to me (I might have guessed more like 30–40, but then this is only a rough estimate, and the spikiness of AI’s abilities makes it rather easy to underestimate this: it causes them to have some abilities sooner, yet to take longer to reach full coverage, so it makes their transition across the human IQ range seem to take longer than if their abilities weren’t spiky). As Sam Altman put it:

"GPT-3 was sort of like talking to a high school student... GPT-4 maybe it was like talking to a college student... [With GPT-5] it's like talking to an expert — a legitimate PhD-level expert in anything, any area you need, on demand."

At that rough exchange rate, Moore’s Law, doubling compute every 2 years, by itself is worth about 7 IQ points a year — not very impressive, though of course it adds up.

This ~50 IQ points per OOM of effective compute number is obviously a rather rough estimate: it could well be off by a factor of two. A much better number could be obtained by looking at the actual tasks used in IRT IQ tests and measuring the widths of the logistic curves for success rates on these across a representative range of AI models (scaled by effective training compute and Epoch Capabilities Index score), compared to the corresponding logistic curve width for humans — this would be a conceptually simple and rather informative research project, but is made more challenging by data contamination of the models having read the answers to standard IQ test questions for which human logistic curve widths are already available (you’d need to come up with or locate unpublished tasks and obtain their saturation curve width across humans). As far as I can tell, no-one has run this experiment properly. The closest thing I could locate was Maxim Lott's TrackingAI project, which (for the subset of tests hopefully avoiding data contamination in the training set) showed an increase of about 15–20 IQ points per year from IQ ~90 in early 2024 to IQ ~125 in early-to-mid 2026. However, measuring this using an entire IQ test battery of questions, rather than IRT on individual questions, is confounded by the spiky profile issue (some items on the test will be comparatively easier for an AI than a human, others harder, so the AI’s score will increase more slowly) — so this is very likely to be an underestimate, and should probably be treated as a lower bound. My best estimate at the moment is probably a broad range: 25–100 IQ points per OOM. For simplicity I'll continue below to use my guesstimated ~50 IQ/OOM number, but bear in mind this is rather a rough estimate, and it might be an overestimate — certainly the effects of spiky AI capabilities tend to make it feel like an overestimate while the AI is anywhere near human level.

This, at least for me, puts a rather different light on Artificial Super-Intelligence (ASI). Temporarily setting aside Recursive Self-Improvement and feedback loops, and similarly setting aside the looming training data, investment, and power walls, if the scaling process of the last 5 years simply continued at the same rate with straight lines on graphs, then ASI is not going to hit, say, IQ 1000 in a few years: at that rate doing that would take more like a couple of decades. Going from IQ 100 to IQ 1000 takes something like 18 orders of magnitude increase in effective compute! (Not to mention roughly 9 orders of magnitude increase in training data volume…) That’s a huge increase, more than enough to start running into physical limits: short of using reversible computation, you hit the Landauer limit on heat dissipation in only about 5 orders of magnitude of processor technology improvement, and silicon-chip-based technology likely maxes out well before that. Limits on algorithmic improvement are harder to estimate, but I would be rather surprised if there were a lot more than about 5 orders of magnitude available there, which would leave us 8 orders of magnitude short. Past those two, you’re looking at scaling up power and resource consumption, and data center power consumption is already of the order of 1% of our total electrical power, so taking this more than a couple of orders of magnitude requires dramatic economic growth.

Eliezer Yudkowsy and Nick Bostrom have both written persuasively about AI blowing right through the human IQ range and keeping going, like an express train speeding through a rural station. On the other hand Scott Alexander has suggested the human range might be good deal wider than this metaphor suggests. I now have a rough estimate of how fast the train's going as it enters the station: something in the region of 50 IQ points per year seems to be a good rough guess. Admittedly, so far we’ve been distilling human intelligence from humans into the AI, and catching up is always easier. Extracting IQ 200 behavior from distilling a body of IQ 50–150 training data sounds hard: we’re likely to need to spend a bunch of compute and effort on generating more and better synthetic training data, across a wide distribution of topics — and depending on the topic, creating that data could be even more compute intensive than training on it. Intelligence is also the logarithm of the amount of training data available, and (outside topics like math and programming where there are clear easily-verifiable rewards) adding more training data generally requires doing actual research, including experiments and data collection. And we need exponentially increasing quantities of data: to sustain an OOM more compute per year you also need over three times as much training data every year, on top of which at least part of that data needs to be of better data quality — by about 50 IQ points smarter. So it’s entirely reasonable to expect the train to slow down somewhat as we hit the data wall: once it gets past the human station, new track needs to be laid for the train to run on, and the compute cost of laying that track increases exponentially.

So realistically, for ASI in, say, the early 2030s, unless we see RSI yielding a lot of speedup in algorithmic improvements and/or Moore’s law, and/or massive economic growth greatly increasing our power production, and we also find that we can mostly extrapolate from subjects where more training data is easy to generate, then we’re likely looking at IQs maybe somewhere in the 300–500 range (to the extent the scale can be meaningfully extrapolated): impressively smarter than any human, but not exactly godlike — heroic or angelic seem more appropriate terms. More likely is that they are IQ 300–500 in mathematics and programming, but not quite as impressive in, say, geology or creative writing or urban planning: topics where rapidly generating vast amounts of new IQ 300–500 quality data is a really challenging problem. (Of course, ASI might also be impressively faster, more parallel, able to spin up copies of itself, have a much larger working memory, have a good intuitive understanding of data modalities that most humans are bad at, or otherwise more capable in ways other than raw IQ — it also might have spiky capabilities that make it in places more or less capable than humans, and might or might not yet have continual learning abilities as good as humans.)

In general, intelligence scaling as the logarithm of the amount of compute (and data) tends to make growth curves look a lot less impressive: exponential growth in compute becomes linear growth in intelligence. It’s really hard work to generate an intelligence explosion if intelligence is the logarithm of compute: there are dramatically decreasing returns on compute. Even superexponential growth of compute from something like recursive self improvement (RSI) can easily end up looking merely superlinear in inteligence, so perhaps polynomial.^[4] Admittedly, a compute curve that actually had a finite-time singularity would still have a finite-time singularity even at a log scale. However, such curves are of course physically impossible: there is actually a physical limit to the amount of compute you can get out of the Solar System. Even a Dyson Swarm of ultratech reversible-computing quantum computronium powered by the sun has a limiting computational capacity.^[5] It is maybe somewhere around 25 orders of magnitude more than our current GPU fleet (before allowing for algorithmic improvements): yet that astronomical ratio is worth maybe 1000–1500 IQ points. So there is an IQ level, probably somewhere around IQ 2000 (depending on credit for algorithmic improvements, reversible computing algorithm design, quantum computing, and so forth), that is actually physically impossible to construct in the solar system.

Now, none of this proves that aligning or controlling ASI will be easy. If something is, say, 100 IQ points smarter than you, then it can easily locate a range of tasks that it can do reliably yet that you will reliably fail on. If you are in conflict with it, and it can pick or create the terrain you’re fighting on, then it can pick a situation where there is a major tactical advantage from solving problems that it can solve and you can’t. So given the choice of terrain or ability to create enough tactical complexity, it can reliably beat you. When comparing two opponents, what matters is the difference in their IQ — don’t look at the ratio between them, that’s not a meaningful way to compare two logarithms. The same is true every time there’s another increase of ~100 IQ points. So IQ maps directly and linearly onto any kind of ELO rank for the outcome of conflicts in which intelligence is relevant. The right way to think about IQ 1000 is that you can create a lineup of ten opponents, with IQs 100, 200, 300, … 800, 900, 1000, where each one will very reliably beat the previous one yet lose to the next one. So even for just a 100 IQ point difference, it becomes essential that we’re not in conflict with the AI. Even ASIs with IQ 250 can likely take over the world if we mess up the alignment problem.

RSI doesn’t need to cause a singularity or intelligence explosion to make the alignment problem much harder. At current rates, adding 100 IQ points, enough to turn AI you might well be able to control into AI that you have basically no chance of controlling, takes about two years — quite a short time compared to the current rate of AI alignment research. Simply accelerating the rate of algorithmic and/or technological and/or economic progress twofold to reduce that to one year, or fourfold down to six months, is more than enough acceleration that it could turn a difficult alignment challenge into something where we simply cannot keep up, unless there was a similar acceleration in alignment research as well (which would obviously be challenging to safely supervise).

On the other hand, it seems plausible that human morality and values, as understood and applied by IQ 50–150 humans, will probably still make sense to an IQ 250 or 300 mind — even if their ability to find loopholes in the letter of the law is ferocious, the spirit of the law seems like it might still look very understandable. So an approach like constitutional AI seems like it might actually still be workable on ASI in that range: if you can make such an ASI still care about human values, those are still likely to look comprehensible. (Indeed, to an ASI with a good understanding of Evolutionary Moral Psychology, they might even seem rather obvious.)

Note that my guesstimated exchange rate of about 50 IQ points to one order of magnitude compute is the ratio for training compute. For inference compute, if using something like the Chinchilla scaling law, it’s the square root of that: a factor of a bit over three in compute for ~50 IQ points (or ~100 IQ points for an order of magnitude in compute). However, that does mean that, as and when we have a genius level (IQ 140) AI, and enough inference compute to run a nation of a million of those, that same amount of compute could instead run a little over 3 million workers with IQ 90 (say doing things like customer service work). Less capable models are cheaper to run, by roughly an order of magnitude per ~100 IQ points. For humans, the smarter ones are rare, and they can reliably do various useful things the less smart ones can’t, so their economic value currently scales up a good deal faster than three-fold per ~50 IQ points. Once we have a wide range of AI, its training compute cost has been paid off, and the market eventually balances, then the economic returns are likely to scale as the actual inference cost ratio — tasks will get routed to models just smart enough to be able to do them reliably, much as most people already do.

1. ^{^}

   Or if the eval has poor answer quality, to nigh-saturated rather than to >90%.

2. ^{^}

   The Epoch Capabilities Index score is constructed off exactly this phenomenon.

3. ^{^}

   This number is harder to estimate and more debated: I’ve seen credible arguments for ranges as wide as – a year, but the combined rate of increase in effective compute is still in the range – per year: i.e. still roughly an order of magnitude per year, just with a bit more uncertainty on the exact rate. Given the very approximate numbers in the rest of my argument, the exact effective compute growth rate makes little difference for my purposes.

4. ^{^}

   Most of the posts and research studies analyzing intelligence explosions, having no good way to estimate the economic gains from IQ 200+ researchers, have instead mostly modeled the fact that a (super)exponentially increasing amount of compute lets you run a (super)exponentially increasing number of genius level (IQ ~140) researchers in parallel (thus incurring a (super)exponentially increasing coordination problem), and then have tried to estimate the likely microeconomic/technological growth consequences. This observation is unquestionably correct, and nothing in this post alters it. These studies have generally treated the new availability of increasingly super-genius researchers as unanalyzable icing on their argument, and hand-waved that it can only speed thing up. This post casts a little more light on both how hard this is and how useful it might be: the supergenius icing lets you solve certain new categories of problem, but it is expensive. You never run out of new smarter flavors of icing, able to solve new categories of problem, but these are exponentially more expensive.

5. ^{^}

   Note that even if one captured the entire power output of the sun, simply lifting the contents of Jupiter and the other gas giants out of their gravity wells takes centuries of that: so there are also hard physical limits on how fast such a system can be created, giving not just a compute maximum but also a rate of increase of compute maximum. It is a general property of the universe that exponentials and superexponentials sooner or later hit limits.