How many OOMs of compute span the human range?
9Raphael Roche
1tickybob
5niplav
3tickybob
4Mo Putera
2niplav
New Comment
Interesting! But "how many 'OOMs of compute' span the human range" should, in my opinion, be the title of the post rather than "how wide is human-level intelligence?". Theoretical compute power is good to know but does not say that much about intelligence. A LLM has – to my limited knowledge – or could have the same neuron count before and after training, but its "intelligence" or capacities rise from null to very high. The capabilities heavily lie in the weights.
It's reasonable to suppose the same for human beings. Neurogenesis occurs almost entirely before birth and neuron count does not change much afterwards. Intelligence probably results largely from the "quality" of the connectivity map that varies genetically and epigenetically from one subject to the other (innate part), and that also comes from their own experience and education, the equivalent of AI training, except it's continuous (acquired part).
In my opinion, the "wideness" of human-level intelligence lies predominantly in the differences between connectivity maps across individuals.
Good post, strong upvoted (though I'd've appreciated citations). The Gaussianity of the human distribution for compute-equivalent seems a relevant crux to me.
Two models:
In one, intelligence may be best modeled as different factors acting in sequence or dependently on another, e.g. the right amount of myelination, number of synapses per neuron, the reuptake speed, the number of cortical columns and just sheer brain volume…; the impact of all of those being multiplied together, if any single one is too low the brain can't function properly and reliable cognition goes to zero. Thus, highly simplified, for some family of random variables . This yields a log-normal (or at least heavy-tailed, if are bounded below) distribution.
In the other, intelligence is the sum of the aforementioned variables: All still contributing to the final performance, but if one is fairly low that's not too bad as other parts can compensate. This aligns well with an infinitesimal model of the genetics of human intelligence, which is widely assumed to be a polygenic trait. Intelligence is a strongly polygenic trait, which under the infinitesimal model implies a normally distributed phenotype, but a significant amount of gene-environment interaction can change that distribution. In this model, , g is normally distributed.
1Thanks for the feedback, exactly the kind of thing I was hoping to get from posting here.
I have thought about a multiplicative model for intelligence, but wouldn't the fact we see pretty-close-to-Gaussian results on intelligence tests tend to disconfirm it? Any residual non-Gaussianity seems like it can be explained by population stratification, etc, rather than a fundamentally non-Gaussian underlying structure. Also, like you say the polygenic evidence also seems to point to a linear additive model being essentially correct.
No, because intelligence test publishers deliberately re-express raw results so their curves have mean = 100 and SD = 15, a convention going back to Wechsler’s “deviation-IQ” idea.
I don't think so, IQ is famously defined to be normally distributed, right?—but we're not interested in convention. I was wondering if there's there some Platonic way in which cognitive ability is naturally distributed between different humans? For example, height is mostly normally distributed, and human lifespan is Gompertz-distributed; it's not very useful to talk about log-height or log-lifespan.
I'm open to the claim that there is no such natural scale for intelligence, or that at least the scale for intelligence is at least similarly natural in some linear and log-scale.
We could go with e.g. RE-bench and see what the distribution over tasks there is for humans...? Or literal program induction? Or ARC-AGI 1/2 and check how the natural distribution for 50% chance of solving a task is.
As for the additive structure of the genetics, my hunch is that it could be the log-transform of some underlying lognormal trait, but I don't know enough about quantitative genetics to identify a flaw in that thinking if it exists.