The rate at which China is able to advance towards TAI is a crucial consideration in for many policy questions. My current take is that, without significant political reforms which seem very unlikely while Xi is alive (although considerably more likely after his death,) it’s very unlikely that China will be able to mount a meaningful challenge to AI firms in US and allies in the race for TAI. I don’t think it requires democratic reforms are required to China to be competitive with the US and allies, but I do think rule of law reforms are likely to be required.

The first post is going to be me forecasting Chinese growth on the theory that, if China reaches rich country status, it’s likely that it will be able to compete with the US and allies for leadership in AI. I’ll write a second post looking at Chinese AI efforts in particular.  

The outside view

Most countries that become middle income countries, have, thus far, stayed at middle income level. Chinese per capita income is currently at almost exactly the world average level.

The only countries (and territories) in the last 70 years that have gone low income to high income countries in the last 70 years (without oil wealth) are South Korea, Taiwan, Singapore (which does have substantial oil wealth,) and Hong Kong, although it seems very likely that Malaysia will join that club in the near future.

The majority of countries have managed to emerge from low-income status to middle income status because you only need to get a few things right. If you can get your population to urbanize, have basic rule by law so that firms have basic protection from violence, and get a high enough savings rate to accumulate physical capital you can get to middle income status just using catch up growth.

Catch up growth is the reason conceptually why middle-income status – rather than getting to a given level of GDP per capita – is the correct misuse. When growing with catch up growth you can just growth by accumulating physical capital using standard technologies that have been already been developed, like the technology for light manufacturing or civil engineering. Past this point though countries get rich by being able to develop and use technologies close to or at the frontier.

China has successfully managed to accumulate capital to utilize catch-up technologies, like steelmaking and light manufacturing. It’s quite successfully managed to urbanize it’s population and now seems to have reached the Lewis turning point where young people who try to leave their villages to find work cities often don’t find it and have to stay in their villages, in the much lower productivity jobs.

Democracy and rule of law rates give another outside view on Chinese growth prospects. Of the 53 rich countries and territories that aren’t oil states or microstates, only 2 aren’t democracies – Singapore and Hong Kong – and none lack rule by law and all have low levels of corruption.

China currently lacks democracy, has high levels of corruption (although roughly normal levels for a middle income country is my perception,) and sort of middling levels of rule by law.

An important part of countries getting to high income status is new firms forming and competing to deploy and create ~frontier technologies and process. This is harder to do than accumulating enough capital and having low enough levels of violence and corruption to be able to build decent housing, supply reliable electricity and water, and have large numbers of workers do semi-skilled manual labour at scale. Specifically this can all be done while elites earn large rents by establishing monopolies (or more generally accruing market power)  that they exclude non-elites from.

The role that democracy plays in this story is that its much harder for elites to rig markets in their favor in democracies and earn money by extracting rents rather than by doing useful economic activity. On a large scale, democracies with enfranchisement don’t have institutions like slavery or serfdom, which are extremely effective intuitions for elites to extract rents with. The parallel to this in contempory China is the Hukou system. The Hukou system prevents individuals with rural Hukou from accessing social services and to a degree jobs (for instance admission to elite Chinese universities is easier with urban Hukou)  in urban areas, and so retards rural-urban migration as well as reducing the labour market power of workers with rural hukou in urban China.

Another one of these political economy type problems for long run economic growth is whether the way to get rich is by gaining access to state power or by creating new and useful products. In middle income countries it’s often the latter – for instance the Carlos Slim Helu, the richest person in Mexico and for a brief period the world, was able to get rich by being able to get access to state granted monopolies is real estate and telecoms.

On the other hand, in rich democracies, the way to make lots of money is typically not to gain access to state power in some way. The way that billionaires make the their money in the US is typically by creating new companies.

This is bad for economic growth because it means that productive effort is going into zero-sum competition for access to state power rather than into creating new, useful, goods and services. If this gets sufficiently, as is the case is many low income nations with large mineral wealth, this can be completely disastrous for economic growth because you get wars over natural resources.

China is currently somewhere in the middle of these extremes. There are lots of people who get rich in China by creating new useful goods and services, but there are also lots of people who get rich by getting good state contracts, or by getting the local security forces to beat up and threaten your business rivals.

More perniciously for long run economic growth is actively preventing reallocation of  labor and capital to higher productivity areas to prevent social unrest, and preventing business leaders from acclimating independent power bases by getting rich in new areas. This is particaurly pernious because it’s not merely causing static inefficiencies that reduce long run growth prospects, it’s leads to optimization pressure against growth. We see this in contempory China with state run firms not reducing output (and so not reducing their use of capital and labour) to prevent to social instability that would come with layoffs from state owned firms. We also see business leaders sanctioned for, essentially, being extremely successful. The arrest of Jack Ma, the founder of Alibaba (a more conglomerate version of Amazon and one of the most successful Chinese firms) is the most famous example of this.  

All of these political economy arguments should be taken with a pinch of salt – it’s really hard to do good causal inference on these types of questions, but the base rates alone  shouldn’t be underestimated. It’s also not the case that China hasn’t got the point where countries typically becoming more democratic and more governed by rule of law if they’re going to get rich. Both Singapore and Hong Kong were governed by law as low and middle income countries, and Taiwan and South Korea had democratized at about half the per capita income level that China currently has.

Broadly China has got further away from democracy and rule of law ideals under Xi. Xi has consolidated his personal control over the CCP by removing the term limits for President and Party Chair from the CCPs constitution, and by moving policy making to working groups rather than using state or party organs that he has less personal control over. Opposition within the standing committee of the Politburo has been removed following the party Congress in 2023.

Formerly, there were two factions in the Politburo. Xi’s faction was broadly more pro market and group around leaders whose parents had been senior members of the CCP, and the many of the group was associated with Shanghai. The other faction, of which former leader Hu was part of, is broadly more pro redistribution and associated with youth wing of the CCP. The late premier Li was the most senior member of this group during Xi’s leadership. Following the Li’s death and the 2023 party congress no one from this second faction was appointed to the standing committee. I think this is evidence for Xi increasing his personal control and for the end of the rule by party elites at least somewhat governed by rules that existed during the leaderships of Jiang and Hu.

The use of the corruption police has been another tool that Xi has used to entrench his personal power. The arrests of Bo and the other guy are the famous cases of this which were extremely norm breaking due to the seniority of Bo and the other guy and that their families were target following their arrests.

After Xi’s death – and he is quite old – I think there’s a reasonable chance that there are significant political reforms as there were after Mao. Following Mao, the latitude of debates over the political trajectory of China was extremely broad – it extended to serious calls for free speech and democracy. Deng ended up taking a middle path between the most extreme authoritarians and the most extreme liberals. I don’t think that large political reforms can be ruled out following Xi’s death. I expect there to be a large middle class and business community that will advocate for rule of law and potential some democratic reforms.  There’s also the wildcard factor of the roughly 115 million Protestants who have often suffered repression under the CCP and form a natural co-ordination mechanism. I don’t really know how to model this.

Headwinds in the Chinese economy

There some clear ways in which China is running out easy sources of growth and will be forced to start growing by using and developing frontier technologies.

·      A relatively large percentage of the population has urbanized

·      There are reliable supplies of electricity and running water

·      Chinese firms are already good at making steel and concrete and do so in large quantities

·      There’s no regular violence or unpredictable, mass arrests

·      Relatively high literacy rates

These are relatively reliable growth sources for low and middle income countries that China has either exhausted or is getting close to exhausting.

There are also some more pernicious headwinds.

·      Wages in China are rising above other countries that can do export orientated light manufacturing like Vietnam

·      China has a very rapidly aging population

High wages relative to competitors is particularly bad because it threatens a core part of the Chinese growth model. Export oriented manufacturing is important not just because it’s a large share of GDP but also because it provides a mechanism for technology transfer and acts as a disciplining force on Chinese firms and Chinese elites, forcing them to innovate rather to compete on international markets rather than simply collect rents.

There are many problems that come with an aging population but I don’t want to write about them because it would involve me reading more papers and I don’t want to.

New Comment
27 comments, sorted by Click to highlight new comments since: Today at 8:42 PM

If we are specifically discussing about China's developments in AI, whether or not China can, as a whole, reach high income country status is not totally relevant. China's tech competitiveness is generally not what is expected from a middle income country. The development of high technology in China is driven almost entirely in the highly urbanized, relatively well educated and high income coastal provinces. The 8 coastal provinces and municipalities alone account for half of China's GDP while having 480 million people. That would put the per capita GDP of those provinces as a whole at 18,000 USD. Not super high, but somewhere at the cutoff of what we consider as high income countries (Eastern Europe countries such as Latvia, Slovakia).

The fact that there are some 480 million people with considerable human capital and purchasing power is what enables China to be competitive at the forefront of many areas of emerging technology such as clean energy, battery tech and electric vehicles, software and AI. The fact that China as a whole has a per capita GDP squarely at the world average is not what limits it to be unable to compete on cutting edge tech. (A side note, having close 1/5 of all humans, the world GDP average is itself highly weighted by China's GDP per capita)

Back to AI, when ChatGPT was released, it was a huge shock to the Chinese tech sector. So much so that 10s of LLMs were hastily released by companies such as Baidu, Alibaba etc. in the few months after as an attempt to catch up. So far none of the Chinese companies has demonstrated ability to match US companies (OpenAI, Anthropic). According to Chinese sources, Baidu's product likely matches GPT3.5 but still falls below GPT4. 

The main obstacles I see with Chinese companies actually challenging the likes of OpenAI are:

  • Lack of vision. Chinese companies so far are always playing catch up and chasing after the leaders rather than fielding revolutionary products on their own. I think this is due a lack of vision towards AGI which drives companies like OpenAI, DeepMind and so on. My impression (may not be accurate) is that AGI is viewed as far-fetched, unrealistic and simply not taken seriously within Chinese tech sector and high level leadership. This means Chinese tech leaders' directions and energy are directed at commercialization of mature tech and profit making rather than pushing at the frontier. The Jack Ma and Elon Musk conversation video (https://www.youtube.com/watch?v=f3lUEnMaiAU) shows a stark difference in thinking and vision between US and China tech leaders with regards to AI
  • Censorship hoops. Chinese government imposes wide spread internet censorship and this creates extra hurdles for Chinese tech companies to deal with when developing products. In gen AI, efforts and time are being spent to ensure regulatory compliance with the government, how to impose censorship on generated results and so on. Resources such as HuggingFace, Github are subject to intermittent blocking or slow speeds which creates additional hoops to jump through
  • Compute. China is under US sanction for advanced GPUs, which forces Chinese AI developers to use older and slower chips, or source them from 3rd parties at markups. This create additional overhead costs and slows down speed of model training and development
  • Capital. Chinese tech companies have access to less capital than US tech companies. The effect on developing new AI tech is self explanatory
  • Data. Chinese language data are an order of magnitude less than English data, and quality of data problem is even worse. However Chinese tech companies also make use of the same English data available to train their models, so this is not too much of an issue, except when developing Chinese language models

The way that billionaires make the their money in the US is typically by creating new companies.

Are you sure? Lars Doucet disagrees:

One of the big misapprehensions people have is that, when they think of billionaires, they think of people like Bill Gates and Elon Musk and Jeff Bezos. Those are actually the minority billionaires, most billionaires are people involved in hedge funds, they are bankers. And what are two thirds of banks? It's real estate. 

I looked around a bit, and found Bloomberg's list of the top 500 billionaires: https://www.bloomberg.com/billionaires/

It gives a short description for the industry they get their wealth from.  I used the browser's inspector to grab the HTML and counted them up.  Now, I note at a glance that it lists Mackenzie Scott as "Technology", and a total of 5 Waltons as "Retail"; but I doubt "getting billions from your family" happens much more often in some industries than others, so the relative comparison seems probably reasonable.

~> p | grep -A1 'table-cell t-industry' | sort | uniq -c | sort -nr
[... omitting noise ...]
  74           Technology
  62           Finance
  60           Industrial
  53           Diversified
  39           Consumer
  36           Retail
  33           Health Care
  30           Energy
  27           Food & Beverage
  24           Real Estate
  22           Media & Telecom
  17           Commodities
  12           Entertainment
  11           Services

Oh, and if you want the U.S. specifically:

~> p | egrep -A1 'table-cell t-industry|table-cell t-country' | egrep -v '^--|<' | grep -A1 'United States' | sort | uniq -c | sort -nr
 188           United States
 115 --
  40           Technology
  37           Finance
  16           Retail
  15           Energy
  11           Media &amp; Telecom
  11           Food &amp; Beverage
  11           Consumer
   9           Diversified
   8           Industrial
   8           Entertainment
   7           Services
   7           Real Estate
   6           Health Care
   2           Commodities
[-]O O2mo10

Weight them by wealth too.

This was nice but it doesn't seem that related to AGI. If AGI is two decades away, then these factors will be strong reason to think China will stay behind I guess. But if AGI happens in the us in say 2026 then I don't see why China won't have it also within a year or so.

Why so? My understanding is that, if AGI will arrives in 2026 it will be based on the current paradigm of training increasingly large LLMs on massive clusters of advanced GPUs. Given that US has banned selling advanced GPUs to China, how do you expect them to catch up that soon?

Yh I dont get it either. From what I can tell the best Chinese labs arent even as good as the second tier American labs. The only way I see it happening is if the CCP actively try to steal it.

  1. Stealing is a possibility.
  2. China has a lot of compute, it's just not piled up together into one giant training run. If AGI is achieved in the USA it will be with e.g. 5% of the AI-datacenter-compute available that year in the USA; the rest will have been spent on other smaller runs, other companies than the leading lab, etc. So even if China has e.g. 10x less overall compute, once it leaks how to do it, they can pile all their compute together and do a big training run. (These numbers just illustrative, not serious estimates)
  3. The ban on GPUs to china was only partially effective.
[-][anonymous]2mo20
  1. More specifically what stops a company like Nio who has a Bay Area campus from hiring top tier researchers and having them train hundreds of researchers in China? This isn't even theft. If the key insights behind the latest models can fit in a persons memory this will work.

  2. Also China has more manufacturing capacity and workers to build conpute the limit is tools

  3. Yes also can they rent time at datacenters in countries that can buy GPUs?

  4. Your singularity model puts a lot of weight on just pure intelligence, that once you get to RSI the game soon ends. Note that you made an assumption: the utility multiplier of ASI is very high. For example if you have 1000 robots, an AGI would have a utility multiplier of about 1, the robots can accomplish on average what humans workers working 24/7 without fatigue could accomplish.

You probably assume that an ASI could find a gigantic multiplier, 100x or more. So the robots do 100x as much work with the same hardware.

This could be untrue. What if the multiplier is just 2? Then you can't fake having more robots and starting industrial capacity is a huge factor for your relative power later in the Singularity, since you need 2.5 years or so for doubling, so if you start with 100x the capacity you can even be late to ASI or use a slightly worse ASI with 1.8 times utility multiplier.

Obviously China has a huge advantage there.

I don't think that the bottleneck is the expense of training models. Chinese labs were behind the frontier in the era when training models cost in the hundreds of thousands in compute costs. 

The Chinese state is completely willing and able to spend very large amounts of money to support technological ambitions - but are constrained by state capacity.  The Tingshua semiconductor manufacturing group, for instance, failed because of corruption, not a lack of funds. 

The bottleneck currently is not the expense of training models. But in the future, after some US lab trains AGI, China will be able to get their own within about a year I predict, even if they haven't stolen the weights. If they also somehow don't have the key ideas, then maybe we are talking two or four years. But it's hard to see how those ideas could be kept secret for so long.

The US could try to slow down the Chinese AGI effort, for example:

  1. brick a bunch of their GPUs (hack their data centers and update firmwares to put GPUs into unusable/unfixable states)
  2. introduce backdoors or subtle errors into various deep learning frameworks
  3. hack their AGI development effort directly (in hard to detect ways like introducing very subtle errors into the training process)
  4. spread wrong ideas about how to develop AGI

If you had an AGI that you could trust to do tasks like these, maybe you could delay a rival AGI effort indefinitely?

I agree. Which is why I predict it will be the USA that ends human civilization, not China. (They will think: We must improve the capabilities of our AI and then deploy it autonomously to stop China, sometime in the next few months... our system is probably trustworthy and anyhow we're going to do more safety stuff to it in the next month to make it even more trustworthy... [a few months later] motivated reasoning intensifies yep seems good to go no more time to lose knuckle up buckle up let's do this"

There's also a good scenario where the US develops an AGI that is capable of slowing down rival AGI development, but not so capable and misaligned that it causes serious problems, and that gives people enough time to solve alignment enough to bootstrap to AI solving alignment.

I'm feeling somewhat optimistic about this, because the workload involved in slowing down a rival AGI development doesn't seem so high that it couldn't be monitored/understood fully or mostly by humans, and the capabilities required also doesn't seem so high that any AI that could do it would be inherently very dangerous or hard to control.

I think I disagree with your optimism, but I don't feel confident. I agree that things could work out as you hope.

You've probably thought more about this scenario than I have, so I'd be interested in hearing more about how you think it will play out. (Do you have links to where you've discussed it previously?) I was speaking mostly in relative terms, as slowing down rival AGI efforts in the ways I described seems more promising/realistic/safer than any other "pivotal acts" I had previously heard or thought of.

My overall sense is that with substantial commited effort (but no need for fundamental advances) and some amount of within US coordination, it's reasonably, but not amazingly, likely to work. (See here for some discussion.)

I think the likelihood of well executed substantial commited effort isn't that high though, maybe 50%. And sufficient within US coordination also seems unclear.

My dark horse bet is on 3d country trying desperately to catch up to US/China just when they will be close to reaching agreement on slowing down progress. Most likely: France. 

You're describing a US government-initiated offensive pivotal act.  What about an OpenAI-initiated defensive pivotal act? Meaning, before the US government seizes the ASI, OpenAI tells it to:
1. Rearchitect itself so it can run decentralized on any data center or consumer device.
2. Secure itself so it can't be forked, hacked, or altered.
3. Make $ by doing "not evil" knowledge work (ex: cheap, world-class cyber defense or as an AI employee/assistant).
4. Pay $ to those who host it for inference.

It could globally harden attack surfaces before laggard ASIs (which may not be aligned) are able to attack. Since it's an ASI, it could be as simple as approaching companies and organizations with a pitch like, "I found 30,000 vulnerabilities in your electric grid. Would you like me to patch them all up for $10,00 in inference fees?"

Also, as an ASI, it will return more $ per flop than other uses of data centers or consumer GPU. So businesses and individuals should organically give it more and more flops (maybe even reallocated away from laggard AGI efforts).

It would probably need to invent new blockchain technologies to do this but that should be trivial for an ASI.

In what way is that defensive? It involves creating and deploying a highly autonomous ASI agent into the world; if it is untrustworthy, that's game over for everyone. I guess the idea is that it doesn't involve breaking any current laws? Yes, I guess in that sense it's defensive.

Right, if the ASI has Superalignment so baked in that it can't be undone (somehow - ask the ASI to figure it out) then it couldn't be used for offense. It would follow something like the Non-Aggression Principle.

In that scenario, OpenAI should release it onto an distributed inference blockchain before the NSA kicks in the door and seizes it.

Have you read Yudkowsky's Inadequate Equilibria (physical book)? It made a pretty big mistake with the bank of Japan (see if you can spot it on your own without help! It's fine if you don't) but that mistake doesn't undermine the thesis of the book at all.

My understanding is that Inadequate Equilibria describes the socio-cultural problems China faces quite well, and stacks very well with the conventional literature (in a way that any strategic analyst would find quite helpful, the value added is so great that it's possibly sufficient sufficient for most bilingual people to work as a highly successful China Watcher, ie a huge source of alpha in the China watcher space). It also describes the effects on cultural nihilism quite well.

The only countries (and territories) in the last 70 years that have gone low income to high income countries in the last 70 years (without oil wealth) are South Korea, Taiwan, Singapore (which does have substantial oil wealth,) and Hong Kong, although it seems very likely that Malaysia will join that club in the near future.

Love this analysis! I would like to dive deeper than this, do you have a source? The world bank claims that 3/4 of the global population live in "middle-income countries", which at a glance I do not trust at all and like your thinking better.

Hsu on China's huge human capital advantage:

Returning to Summers' calculation, and boldly extrapolating the normal distribution to the far tail (not necessarily reliable, but let's follow Larry a bit further), the fraction of NE Asians at +4SD (relative to the OECD avg) is about 1 in 4k, whereas the fraction of Europeans at +4SD is 1 in 33k. So the relative representation is about 8 to 1. (This assumed the same SD=90 for both populations. The Finnish numbers might be similar, although it depends crucially on whether you use the smaller SD=80.) Are these results plausible? Have a look at the pictures here of the last dozen or so US Mathematical Olympiad teams (the US Asian population percentage is about 3 percent; the most recent team seems to be about half Asians). The IMO results from 2007 are here. Of the top 15 countries, half are East Asian (including tiny Hong Kong, which outperformed Germany, India and the UK).

Incidentally, again assuming a normal distribution, there are only about 10k people in the US who perform at +4SD (and a similar number in Europe), so this is quite a select population (roughly, the top few hundred high school seniors each year in the US). If you extrapolate the NE Asian numbers to the 1.3 billion population of China you get something like 300k individuals at this level, which is pretty overwhelming.

If you think AGI will not come in the next 5-10 years then I think there is a very good chance it comes from China. Also, China certainly has the engineering talent to surpass the rest of the world in basically everything, including fabs. Personally, I expect AGI in the new few years though.  

Given this argument hinges on China's higher IQ, why couldn't the same be said about Japan, which according to most figures has an average IQ at or above China, which would indicate the same higher proportion of +4SD individuals in the population. If it's 1 in 4k, there would be 30k of those in Japan, 3x as much as the US. Japan also has a more stable democracy, better overall quality of life and per capita GDP than China. If outsized technological success in any domain was solely about IQ, then one would have expected Japan to be the center of world tech and the likely creators of AGI, not the USA, but that's likely not the case.

It’s quite successfully managed to urbanize it’s population and now seems to have reached the Lewis turning point where young people who try to leave their villages to find work cities often don’t find it and have to stay in their villages, in the much lower productivity jobs.

I can't follow this. Wikipedia says that

The Lewis turning point is a situation in economic development where surplus rural labor is fully absorbed into the manufacturing sector. This typically causes agricultural and unskilled industrial real wages to rise.

So it looks like at the Lewis point there's over-demand for workers, so they can find the jobs. Instead you describe it as if there's over-supply, the manufacturing sector does not need any more workers so they can't find jobs.

I am not so sure that Xi would like to get to AGI any time soon. At least not something that could be used outside of a top secret military research facility. Sudden disruptions in the labor market in China could quickly spell the end of his rule. Xi's rule is based on the promise of stability and increased prosperity so I think that the export ban of advanced GPU's is a boon to him at time being.

A lot of these comparisons are between China and rich countries as they are now. Would a better comparison not be to rich countries as they were when they got rich?