If we are specifically discussing about China's developments in AI, whether or not China can, as a whole, reach high income country status is not totally relevant. China's tech competitiveness is generally not what is expected from a middle income country. The development of high technology in China is driven almost entirely in the highly urbanized, relatively well educated and high income coastal provinces. The 8 coastal provinces and municipalities alone account for half of China's GDP while having 480 million people. That would put the per capita GDP of those provinces as a whole at 18,000 USD. Not super high, but somewhere at the cutoff of what we consider as high income countries (Eastern Europe countries such as Latvia, Slovakia).
The fact that there are some 480 million people with considerable human capital and purchasing power is what enables China to be competitive at the forefront of many areas of emerging technology such as clean energy, battery tech and electric vehicles, software and AI. The fact that China as a whole has a per capita GDP squarely at the world average is not what limits it to be unable to compete on cutting edge tech. (A side note, having close 1/5 of all humans, the world GDP average is itself highly weighted by China's GDP per capita)
Back to AI, when ChatGPT was released, it was a huge shock to the Chinese tech sector. So much so that 10s of LLMs were hastily released by companies such as Baidu, Alibaba etc. in the few months after as an attempt to catch up. So far none of the Chinese companies has demonstrated ability to match US companies (OpenAI, Anthropic). According to Chinese sources, Baidu's product likely matches GPT3.5 but still falls below GPT4.
The main obstacles I see with Chinese companies actually challenging the likes of OpenAI are:
The way that billionaires make the their money in the US is typically by creating new companies.
Are you sure? Lars Doucet disagrees:
One of the big misapprehensions people have is that, when they think of billionaires, they think of people like Bill Gates and Elon Musk and Jeff Bezos. Those are actually the minority billionaires, most billionaires are people involved in hedge funds, they are bankers. And what are two thirds of banks? It's real estate.
I looked around a bit, and found Bloomberg's list of the top 500 billionaires: https://www.bloomberg.com/billionaires/
It gives a short description for the industry they get their wealth from. I used the browser's inspector to grab the HTML and counted them up. Now, I note at a glance that it lists Mackenzie Scott as "Technology", and a total of 5 Waltons as "Retail"; but I doubt "getting billions from your family" happens much more often in some industries than others, so the relative comparison seems probably reasonable.
~> p | grep -A1 'table-cell t-industry' | sort | uniq -c | sort -nr
[... omitting noise ...]
74 Technology
62 Finance
60 Industrial
53 Diversified
39 Consumer
36 Retail
33 Health Care
30 Energy
27 Food & Beverage
24 Real Estate
22 Media & Telecom
17 Commodities
12 Entertainment
11 ServicesOh, and if you want the U.S. specifically:
~> p | egrep -A1 'table-cell t-industry|table-cell t-country' | egrep -v '^--|<' | grep -A1 'United States' | sort | uniq -c | sort -nr
188 United States
115 --
40 Technology
37 Finance
16 Retail
15 Energy
11 Media & Telecom
11 Food & Beverage
11 Consumer
9 Diversified
8 Industrial
8 Entertainment
7 Services
7 Real Estate
6 Health Care
2 CommoditiesThis was nice but it doesn't seem that related to AGI. If AGI is two decades away, then these factors will be strong reason to think China will stay behind I guess. But if AGI happens in the us in say 2026 then I don't see why China won't have it also within a year or so.
Why so? My understanding is that, if AGI will arrives in 2026 it will be based on the current paradigm of training increasingly large LLMs on massive clusters of advanced GPUs. Given that US has banned selling advanced GPUs to China, how do you expect them to catch up that soon?
Yh I dont get it either. From what I can tell the best Chinese labs arent even as good as the second tier American labs. The only way I see it happening is if the CCP actively try to steal it.
More specifically what stops a company like Nio who has a Bay Area campus from hiring top tier researchers and having them train hundreds of researchers in China? This isn't even theft. If the key insights behind the latest models can fit in a persons memory this will work.
Also China has more manufacturing capacity and workers to build conpute the limit is tools
Yes also can they rent time at datacenters in countries that can buy GPUs?
Your singularity model puts a lot of weight on just pure intelligence, that once you get to RSI the game soon ends. Note that you made an assumption: the utility multiplier of ASI is very high. For example if you have 1000 robots, an AGI would have a utility multiplier of about 1, the robots can accomplish on average what humans workers working 24/7 without fatigue could accomplish.
You probably assume that an ASI could find a gigantic multiplier, 100x or more. So the robots do 100x as much work with the same hardware.
This could be untrue. What if the multiplier is just 2? Then you can't fake having more robots and starting industrial capacity is a huge factor for your relative power later in the Singularity, since you need 2.5 years or so for doubling, so if you start with 100x the capacity you can even be late to ASI or use a slightly worse ASI with 1.8 times utility multiplier.
Obviously China has a huge advantage there.
I don't think that the bottleneck is the expense of training models. Chinese labs were behind the frontier in the era when training models cost in the hundreds of thousands in compute costs.
The Chinese state is completely willing and able to spend very large amounts of money to support technological ambitions - but are constrained by state capacity. The Tingshua semiconductor manufacturing group, for instance, failed because of corruption, not a lack of funds.
The bottleneck currently is not the expense of training models. But in the future, after some US lab trains AGI, China will be able to get their own within about a year I predict, even if they haven't stolen the weights. If they also somehow don't have the key ideas, then maybe we are talking two or four years. But it's hard to see how those ideas could be kept secret for so long.
The US could try to slow down the Chinese AGI effort, for example:
If you had an AGI that you could trust to do tasks like these, maybe you could delay a rival AGI effort indefinitely?
I agree. Which is why I predict it will be the USA that ends human civilization, not China. (They will think: We must improve the capabilities of our AI and then deploy it autonomously to stop China, sometime in the next few months... our system is probably trustworthy and anyhow we're going to do more safety stuff to it in the next month to make it even more trustworthy... [a few months later] motivated reasoning intensifies yep seems good to go no more time to lose knuckle up buckle up let's do this"
My dark horse bet is on 3d country trying desperately to catch up to US/China just when they will be close to reaching agreement on slowing down progress. Most likely: France.
There's also a good scenario where the US develops an AGI that is capable of slowing down rival AGI development, but not so capable and misaligned that it causes serious problems, and that gives people enough time to solve alignment enough to bootstrap to AI solving alignment.
I'm feeling somewhat optimistic about this, because the workload involved in slowing down a rival AGI development doesn't seem so high that it couldn't be monitored/understood fully or mostly by humans, and the capabilities required also doesn't seem so high that any AI that could do it would be inherently very dangerous or hard to control.
I think I disagree with your optimism, but I don't feel confident. I agree that things could work out as you hope.
You've probably thought more about this scenario than I have, so I'd be interested in hearing more about how you think it will play out. (Do you have links to where you've discussed it previously?) I was speaking mostly in relative terms, as slowing down rival AGI efforts in the ways I described seems more promising/realistic/safer than any other "pivotal acts" I had previously heard or thought of.
My overall sense is that with substantial commited effort (but no need for fundamental advances) and some amount of within US coordination, it's reasonably, but not amazingly, likely to work. (See here for some discussion.)
I think the likelihood of well executed substantial commited effort isn't that high though, maybe 50%. And sufficient within US coordination also seems unclear.
You're describing a US government-initiated offensive pivotal act. What about an OpenAI-initiated defensive pivotal act? Meaning, before the US government seizes the ASI, OpenAI tells it to:
1. Rearchitect itself so it can run decentralized on any data center or consumer device.
2. Secure itself so it can't be forked, hacked, or altered.
3. Make $ by doing "not evil" knowledge work (ex: cheap, world-class cyber defense or as an AI employee/assistant).
4. Pay $ to those who host it for inference.
It could globally harden attack surfaces before laggard ASIs (which may not be aligned) are able to attack. Since it's an ASI, it could be as simple as approaching companies and organizations with a pitch like, "I found 30,000 vulnerabilities in your electric grid. Would you like me to patch them all up for $10,00 in inference fees?"
Also, as an ASI, it will return more $ per flop than other uses of data centers or consumer GPU. So businesses and individuals should organically give it more and more flops (maybe even reallocated away from laggard AGI efforts).
It would probably need to invent new blockchain technologies to do this but that should be trivial for an ASI.
In what way is that defensive? It involves creating and deploying a highly autonomous ASI agent into the world; if it is untrustworthy, that's game over for everyone. I guess the idea is that it doesn't involve breaking any current laws? Yes, I guess in that sense it's defensive.
Right, if the ASI has Superalignment so baked in that it can't be undone (somehow - ask the ASI to figure it out) then it couldn't be used for offense. It would follow something like the Non-Aggression Principle.
In that scenario, OpenAI should release it onto an distributed inference blockchain before the NSA kicks in the door and seizes it.
Have you read Yudkowsky's Inadequate Equilibria (physical book)? It made a pretty big mistake with the bank of Japan (see if you can spot it on your own without help! It's fine if you don't) but that mistake doesn't undermine the thesis of the book at all.
My understanding is that Inadequate Equilibria describes the socio-cultural problems China faces quite well, and stacks very well with the conventional literature (in a way that any strategic analyst would find quite helpful, the value added is so great that it's possibly sufficient sufficient for most bilingual people to work as a highly successful China Watcher, ie a huge source of alpha in the China watcher space). It also describes the effects on cultural nihilism quite well.
The only countries (and territories) in the last 70 years that have gone low income to high income countries in the last 70 years (without oil wealth) are South Korea, Taiwan, Singapore (which does have substantial oil wealth,) and Hong Kong, although it seems very likely that Malaysia will join that club in the near future.
Love this analysis! I would like to dive deeper than this, do you have a source? The world bank claims that 3/4 of the global population live in "middle-income countries", which at a glance I do not trust at all and like your thinking better.
Hsu on China's huge human capital advantage:
Returning to Summers' calculation, and boldly extrapolating the normal distribution to the far tail (not necessarily reliable, but let's follow Larry a bit further), the fraction of NE Asians at +4SD (relative to the OECD avg) is about 1 in 4k, whereas the fraction of Europeans at +4SD is 1 in 33k. So the relative representation is about 8 to 1. (This assumed the same SD=90 for both populations. The Finnish numbers might be similar, although it depends crucially on whether you use the smaller SD=80.) Are these results plausible? Have a look at the pictures here of the last dozen or so US Mathematical Olympiad teams (the US Asian population percentage is about 3 percent; the most recent team seems to be about half Asians). The IMO results from 2007 are here. Of the top 15 countries, half are East Asian (including tiny Hong Kong, which outperformed Germany, India and the UK).
Incidentally, again assuming a normal distribution, there are only about 10k people in the US who perform at +4SD (and a similar number in Europe), so this is quite a select population (roughly, the top few hundred high school seniors each year in the US). If you extrapolate the NE Asian numbers to the 1.3 billion population of China you get something like 300k individuals at this level, which is pretty overwhelming.
If you think AGI will not come in the next 5-10 years then I think there is a very good chance it comes from China. Also, China certainly has the engineering talent to surpass the rest of the world in basically everything, including fabs. Personally, I expect AGI in the new few years though.
Given this argument hinges on China's higher IQ, why couldn't the same be said about Japan, which according to most figures has an average IQ at or above China, which would indicate the same higher proportion of +4SD individuals in the population. If it's 1 in 4k, there would be 30k of those in Japan, 3x as much as the US. Japan also has a more stable democracy, better overall quality of life and per capita GDP than China. If outsized technological success in any domain was solely about IQ, then one would have expected Japan to be the center of world tech and the likely creators of AGI, not the USA, but that's likely not the case.
It’s quite successfully managed to urbanize it’s population and now seems to have reached the Lewis turning point where young people who try to leave their villages to find work cities often don’t find it and have to stay in their villages, in the much lower productivity jobs.
I can't follow this. Wikipedia says that
The Lewis turning point is a situation in economic development where surplus rural labor is fully absorbed into the manufacturing sector. This typically causes agricultural and unskilled industrial real wages to rise.
So it looks like at the Lewis point there's over-demand for workers, so they can find the jobs. Instead you describe it as if there's over-supply, the manufacturing sector does not need any more workers so they can't find jobs.
I am not so sure that Xi would like to get to AGI any time soon. At least not something that could be used outside of a top secret military research facility. Sudden disruptions in the labor market in China could quickly spell the end of his rule. Xi's rule is based on the promise of stability and increased prosperity so I think that the export ban of advanced GPU's is a boon to him at time being.
A lot of these comparisons are between China and rich countries as they are now. Would a better comparison not be to rich countries as they were when they got rich?
The rate at which China is able to advance towards TAI is a crucial consideration in for many policy questions. My current take is that, without significant political reforms which seem very unlikely while Xi is alive (although considerably more likely after his death,) it’s very unlikely that China will be able to mount a meaningful challenge to AI firms in US and allies in the race for TAI. I don’t think it requires democratic reforms are required to China to be competitive with the US and allies, but I do think rule of law reforms are likely to be required.
The first post is going to be me forecasting Chinese growth on the theory that, if China reaches rich country status, it’s likely that it will be able to compete with the US and allies for leadership in AI. I’ll write a second post looking at Chinese AI efforts in particular.
The outside view
Most countries that become middle income countries, have, thus far, stayed at middle income level. Chinese per capita income is currently at almost exactly the world average level.
The only countries (and territories) in the last 70 years that have gone low income to high income countries in the last 70 years (without oil wealth) are South Korea, Taiwan, Singapore (which does have substantial oil wealth,) and Hong Kong, although it seems very likely that Malaysia will join that club in the near future.
The majority of countries have managed to emerge from low-income status to middle income status because you only need to get a few things right. If you can get your population to urbanize, have basic rule by law so that firms have basic protection from violence, and get a high enough savings rate to accumulate physical capital you can get to middle income status just using catch up growth.
Catch up growth is the reason conceptually why middle-income status – rather than getting to a given level of GDP per capita – is the correct misuse. When growing with catch up growth you can just growth by accumulating physical capital using standard technologies that have been already been developed, like the technology for light manufacturing or civil engineering. Past this point though countries get rich by being able to develop and use technologies close to or at the frontier.
China has successfully managed to accumulate capital to utilize catch-up technologies, like steelmaking and light manufacturing. It’s quite successfully managed to urbanize it’s population and now seems to have reached the Lewis turning point where young people who try to leave their villages to find work cities often don’t find it and have to stay in their villages, in the much lower productivity jobs.
Democracy and rule of law rates give another outside view on Chinese growth prospects. Of the 53 rich countries and territories that aren’t oil states or microstates, only 2 aren’t democracies – Singapore and Hong Kong – and none lack rule by law and all have low levels of corruption.
China currently lacks democracy, has high levels of corruption (although roughly normal levels for a middle income country is my perception,) and sort of middling levels of rule by law.
An important part of countries getting to high income status is new firms forming and competing to deploy and create ~frontier technologies and process. This is harder to do than accumulating enough capital and having low enough levels of violence and corruption to be able to build decent housing, supply reliable electricity and water, and have large numbers of workers do semi-skilled manual labour at scale. Specifically this can all be done while elites earn large rents by establishing monopolies (or more generally accruing market power) that they exclude non-elites from.
The role that democracy plays in this story is that its much harder for elites to rig markets in their favor in democracies and earn money by extracting rents rather than by doing useful economic activity. On a large scale, democracies with enfranchisement don’t have institutions like slavery or serfdom, which are extremely effective intuitions for elites to extract rents with. The parallel to this in contempory China is the Hukou system. The Hukou system prevents individuals with rural Hukou from accessing social services and to a degree jobs (for instance admission to elite Chinese universities is easier with urban Hukou) in urban areas, and so retards rural-urban migration as well as reducing the labour market power of workers with rural hukou in urban China.
Another one of these political economy type problems for long run economic growth is whether the way to get rich is by gaining access to state power or by creating new and useful products. In middle income countries it’s often the latter – for instance the Carlos Slim Helu, the richest person in Mexico and for a brief period the world, was able to get rich by being able to get access to state granted monopolies is real estate and telecoms.
On the other hand, in rich democracies, the way to make lots of money is typically not to gain access to state power in some way. The way that billionaires make the their money in the US is typically by creating new companies.
This is bad for economic growth because it means that productive effort is going into zero-sum competition for access to state power rather than into creating new, useful, goods and services. If this gets sufficiently, as is the case is many low income nations with large mineral wealth, this can be completely disastrous for economic growth because you get wars over natural resources.
China is currently somewhere in the middle of these extremes. There are lots of people who get rich in China by creating new useful goods and services, but there are also lots of people who get rich by getting good state contracts, or by getting the local security forces to beat up and threaten your business rivals.
More perniciously for long run economic growth is actively preventing reallocation of labor and capital to higher productivity areas to prevent social unrest, and preventing business leaders from acclimating independent power bases by getting rich in new areas. This is particaurly pernious because it’s not merely causing static inefficiencies that reduce long run growth prospects, it’s leads to optimization pressure against growth. We see this in contempory China with state run firms not reducing output (and so not reducing their use of capital and labour) to prevent to social instability that would come with layoffs from state owned firms. We also see business leaders sanctioned for, essentially, being extremely successful. The arrest of Jack Ma, the founder of Alibaba (a more conglomerate version of Amazon and one of the most successful Chinese firms) is the most famous example of this.
All of these political economy arguments should be taken with a pinch of salt – it’s really hard to do good causal inference on these types of questions, but the base rates alone shouldn’t be underestimated. It’s also not the case that China hasn’t got the point where countries typically becoming more democratic and more governed by rule of law if they’re going to get rich. Both Singapore and Hong Kong were governed by law as low and middle income countries, and Taiwan and South Korea had democratized at about half the per capita income level that China currently has.
Broadly China has got further away from democracy and rule of law ideals under Xi. Xi has consolidated his personal control over the CCP by removing the term limits for President and Party Chair from the CCPs constitution, and by moving policy making to working groups rather than using state or party organs that he has less personal control over. Opposition within the standing committee of the Politburo has been removed following the party Congress in 2023.
Formerly, there were two factions in the Politburo. Xi’s faction was broadly more pro market and group around leaders whose parents had been senior members of the CCP, and the many of the group was associated with Shanghai. The other faction, of which former leader Hu was part of, is broadly more pro redistribution and associated with youth wing of the CCP. The late premier Li was the most senior member of this group during Xi’s leadership. Following the Li’s death and the 2023 party congress no one from this second faction was appointed to the standing committee. I think this is evidence for Xi increasing his personal control and for the end of the rule by party elites at least somewhat governed by rules that existed during the leaderships of Jiang and Hu.
The use of the corruption police has been another tool that Xi has used to entrench his personal power. The arrests of Bo and the other guy are the famous cases of this which were extremely norm breaking due to the seniority of Bo and the other guy and that their families were target following their arrests.
After Xi’s death – and he is quite old – I think there’s a reasonable chance that there are significant political reforms as there were after Mao. Following Mao, the latitude of debates over the political trajectory of China was extremely broad – it extended to serious calls for free speech and democracy. Deng ended up taking a middle path between the most extreme authoritarians and the most extreme liberals. I don’t think that large political reforms can be ruled out following Xi’s death. I expect there to be a large middle class and business community that will advocate for rule of law and potential some democratic reforms. There’s also the wildcard factor of the roughly 115 million Protestants who have often suffered repression under the CCP and form a natural co-ordination mechanism. I don’t really know how to model this.
Headwinds in the Chinese economy
There some clear ways in which China is running out easy sources of growth and will be forced to start growing by using and developing frontier technologies.
· A relatively large percentage of the population has urbanized
· There are reliable supplies of electricity and running water
· Chinese firms are already good at making steel and concrete and do so in large quantities
· There’s no regular violence or unpredictable, mass arrests
· Relatively high literacy rates
These are relatively reliable growth sources for low and middle income countries that China has either exhausted or is getting close to exhausting.
There are also some more pernicious headwinds.
· Wages in China are rising above other countries that can do export orientated light manufacturing like Vietnam
· China has a very rapidly aging population
High wages relative to competitors is particularly bad because it threatens a core part of the Chinese growth model. Export oriented manufacturing is important not just because it’s a large share of GDP but also because it provides a mechanism for technology transfer and acts as a disciplining force on Chinese firms and Chinese elites, forcing them to innovate rather to compete on international markets rather than simply collect rents.
There are many problems that come with an aging population but I don’t want to write about them because it would involve me reading more papers and I don’t want to.