I partially agree, and I think this is a great point! I do think that there will be significantly accelerated economic growth; prestige economies (i.e., hand-crafted luxury goods, personal-taste software like video games, personal interaction, taste-based goods (i.e., somewhat arbitrary values), silly things like "hand-watered gardens") tend to produce considerably higher incomes than traditional industrial (let alone agrarian) economies. So rather than "economic growth stays the same", we might see a rapid transition towards a post-necessity economy (post-scarcity in some ways). But it's true that mass growth of raw industrial goods and services does not correspond with the complete replacement of an economy's other sectors, regardless of quality.
That said, historically speaking, most new added-value jobs tend to augment the productivity of the previously "essential" sectors of the economy: factories produced shirts and hoes and tractors and fertilizers and such that bolstered the agrarian economy very greatly, and in turn the services economy such as banking, telecomms, software engineering (programmable logic controllers!) etc. bolstered industrial production and efficiency very much. The new prestige economy will not be that; VTubers probably won't enhance the productivity of AGIs, so there'll certainly be significant differences compared to historical changes, which I imagine is the point imagined by ASI-pilled people in AI 2040 and such.
Which is why I disagree on a small point: countries that do not engage in such labor, for that reason, probably won't end up as third-world countries but probably second-world countries (high-developing to low-developed countries like China, Malaysia, Romania, Turkey, Brazil, Kazakhstan, maybe Poland, etc.). Enough that there'll be employment, but probably as a choice rather than a necessity, varying between societies, cultures, and personal aspirations.
While that is true, certainly human alignment researchers will not be enough to constitute an economically vital force of their own? I mean, I can see it being potentially like the software engineering industry was a few years ago; who would have thought that "programming computers" would be a major job if you asked people in 1926. And even average humans have shown the propensity to be able to operate extremely sophisticated systems, but it does seem something of a reach that we'd have a demographically and economically significant population of human ali... (read more)
The AI futures project predicts that under plan A
and
I think this will be false, but for somewhat boring reasons: the prices in the robot economy will fall exponentially to counteract this.
Of course, prices are relative! However, assuming there are some jobs that require humans for non-economic reasons (such as goods produced by the Amish, who refrain from using electricity for religious reasons), the price of those jobs will be relatively stable. And if the result of the jobs are automated, the rest of humanity needs to either get one of these jobs or retire. For example, I think the vast majority of the human workforce would "need" to go into non-AI art, chess competitions, etc... I put "need" in scare quotes because you could obviously survive without these things, but the standard of living would probably rise to include them. This is the same reason humans need to work despite agriculture being mostly automated: consumerism filled the void that physical hunger left, creating an abundance of factory jobs. Counterintuitively, with AGI it would be possible for entire countries to physically survive with no human labor, but they would fall behind GDP wise and be considered third world countries. I call this the newly "needed" labor. I'm not the first person to think about this kind of labor (see this Dwarkesh podcast which gives the example of ballerinas) but I haven't seen it specifically applied to AI 2040.
So anyways, I predict that most people will be thinking of prices compared to these activities, since they are more expensive and tied to wages. Purely economic labor, both physical and cognitive, decays exponentially in price. Kind of like solar panels, but more extreme (it will almost exactly cancel in the case of AGI):
The reason this is somewhat boring is that this doesn't change the recursive self improvement math. RSI is purely economic, so what happens is that the balance sheet of the AI companies "look normal" but represent huge amounts of hardware and technology in an absolute, non-human sense. For example, OpenAI's balance sheet might show that they are "only" spending a billion dollars per year buying chips, but each year they are buying twice as many chips at half the price, because they are using robots that are half the price as last year, being built by robots that are half the price as last year, etc...
But I think precision in writing and predict still matters! So I'd suggest that AI 2040 revise their economic prediction to be that the balance sheets and economic growth look normal, but that there will be an explosive fall in the prices of robots and chips and all other purely economic activity, matching AI 2040's current predictions for the physical number of chips and robots (i.e. that they grow explosively). Otherwise, I think the economic predictions will be useful but technically incorrect.
This prediction is also more compatible with mainstream economic thinking while still predicting explosive RSI! In particular, I think it would make it easier to work with an economist to predict the number of robots if you tell them the price of robots is going down instead of that the entire economy is going up.