AI improvements to date may come from picking low-hanging fruit. It can’t do math reliably? Let it use a calculator. It improves with more parameters? Scale it up and see if it helps even more.

These improvements rely on the availability of significant, well-defined problems with concrete solutions.

As these polite problems are solved, we may find that vendors find that the issues user report are increasingly hard to define, do not have clear solutions, or only have solutions that entail significant tradeoffs.

The rush by companies to deploy massive capex to scale further despite how the model training horse race destroys profitability supports this hypothesis. If they had cheap alternative ways to scale, they’d be prioritizing that. They may be out of other ideas and hoping that spending $ will lead to a magical, mysterious AI-guided ultimate victory. Hence, they call it a “scaling law,” instead of a “recently observed historic trend.”

Supply is elastic. It may also be that companies will discover new concrete and cheap dimensions of improvement as they scale, and as AI adoption continues and as production practices are reoriented around AI and anticipation of AI improvements.

But we do see true asymptotes in other industries. In transport, drivers and airline passengers are not getting around appreciably faster now than they were in 1980. The most we hope for is supersonic.

Food is much better than it was in the past, but nobody is expecting that food will eventually be ten times tastier and more nutritious than it is now due to innovation in food technology.

Shipping was revolutionized by containers and improved by guidance technologies, but again, nobody is anticipating dramatic improvements in that sector either.

Video games have improved, but it seems clear that the obvious dimensions of improvement have been mostly explored, and that innovations now appeal primarily to niche interests due to innovations in game markets and engines that enable those niche interests to be satisfied at low cost. Movies, music, art, fashion and literature have had more time to mature and are even more widely recognized as having arrived at some sort of asymptote.

Construction, an ancient industry, continues to improve slowly, with new materials and tools and manufacturing methods and regulatory practices. Perhaps because this industry is so diverse and because buildings and building methods must satisfy such a diverse range of requirements both from consumers and from producers, there are enough dimensions of improvement to keep progress going even after all this time.

Moore’s Law is slowing down. But GPUs seem to continue to undergo fast-paced innovation, and trade secrets still seem critical to the industry, which I would not expect in an industry that has saturated its space for progress.

In energy, there is clearly plenty of room for innovation, from batteries, to solar and wind, to nuclear, to fission. Slow growth in this sector in nuclear and fission seems to be primarily due to regulation and lack of research funding.

Biomedical advancements continue. Like energy, the slow pace of conversion into products is primarily attributable to regulation/political punishment, not exhaustion of the innovation search space.

We can see that while saturation of innovation opportunities is real, industries can be ancient (construction) and yet continue to find meaningful ways to innovate. They can also be novel (movies and video games) and yet exhibit signs of exhaustion. Innovation can slow due to exhaustion, regulation, or lack of investment.

It seem superficially unwise to assume a technology as novel as AI has reached saturation. However, our deeply limited theory on what “intelligence” is, and how to improve it, sets it apart from other engineering disciplines as well as from computer science generally. It puts it in some ways closer to fields like management, sociology and politics (where we struggle to define what exactly it is we’re studying) and also to fields like biology (where mechanisms are complex and hard to tease apart, but where we can at least try to run controlled experiments and have at least some theory to guide their design).

My guess is that progress in AI will follow a trajectory analogous to progress in medicine. There was a sharp break in the late 1800s as concepts of sanitation and germ theory and chemistry took hold, leading to a sudden and rapid picking of low hanging fruit, a general excitement about the possibilities of progress, an embrace of the “radical,” a public fascination.

Then the low hanging fruit was exhausted. Despite the continuous and extraordinary improvements in methods and understanding that could have been seen as trending toward “self-improvement” and acceleration and “takeoff,“ this isn’t what we see.

Instead, limited ability to probe, predict and intervene on complex biological systems over relevant time and population scales has made every inch of progress hard won. We cannot just experiment on people. We have struggled to produce quality models. We have discovered one dimension of profound challenge after another, as hype cycle has followed hype cycle in the quest for immortality, a quest that’s just as grandiose as the quest for AGI.

AGI will face its own analogous challenges. It is so expensive to train and run models that you can’t freely and infinitely run useful experiments on this process. We lack the equivalent for intelligence of germ theory, evolution, chemistry and anatomy: we can’t break down intelligence into component parts to understand their function and relationship to the whole. We cannot identify the equivalent of “the DNA of intelligence“ or explain hallucinations the way we can explain heart attacks.

AI innovation has advantages that medicine lacks: it is operating on machines, so there are fewer ethical and legal barriers to experiment, and it is running on hardware and software that can be directly inspected and precisely modified down to the level of bits.

It’s unclear to me how the advantages and disadvantages of AI will play out in terms of progress rates over time. I‘m confident AI is not at some fundamental asymptote of possible progress. But every other industry I can think of has transitioned from an early boom in innovation to a long grinding slow climb in capabilities over a century or more.

i understand the objection that unlike other technologies, which are capital, AI can act as and improve on and replace labor. That the capital/labor division is not a fundamental truth, just categories of convenience that have exhausted their utility.

But that’s not really what I’m arguing against. I just think it might still take a long time to go from where AI is now to it having the full complement of human capabilities that would make it a genuine, general substitute for human labor. We don’t actually know what those capabilities are. We are actively discovering our differentiators as we go.

In the past, we thought that certain capabilities like artistic creativity or chess were distinctively human. We’ve seen AI demolish those fantasies.

But we have also seen that human capabilities a computer could easily replicate, like demonstrating awareness of what it does not know, or holding a consistent opinion, or recognizing when you’re going in circles, are actually difficult for AI. And we don’t know why. Only with advancing capabilities do we discover what’s truly easy or hard for a computer.

We also haven’t adequately considered the possibility that as capabilities advance, the capabilities requirements to produce further gains may also advance. If it takes a team of brilliant world-class human AI experts to produce current marginal AI capabilities gains, what if there comes a pre-full-AGI day when neither they nor whatever the frontier models are at that time are capable of eking out substantial further improvements in capabilities? It may be too hard for humans plus subhuman AI to make a human-level AI.

This is only a hypothesis, but it leaves some real room for doubt as to where AI will be in the next 30 or even 100 years. I might very well be an old man who lives in a world full of exceptional AI-produced software tools for processing information and where factories or corporations are managed autonomously by AI, and yet which resembles this world about as much as this world resembles 1975.