When I write “China”, I refer to the political and economic entity, the People’s Republic of China, founded in 1949.

Leading US AI companies are currently rushing towards developing artificial general intelligence (AGI) by building and training increasingly powerful large language models (LLMs), as well as building architecture on top of them. This is frequently framed in terms of a great power competition for technological supremacy between the US and China.

However, China has neither the resources nor any interest in competing with the US in developing artificial general intelligence (AGI) primarily via scaling Large Language Models (LLMs).

In brief, China does not compete with the US in developing AGI via LLMs because of:

Resources and technology: China does not have access to the computational resources^[1] (compute, here specifically data centre-grade GPUs) needed for large-scale training runs of large language models. This gap will only widen over time; China is failing to develop a domestic semiconductor industry, despite massive efforts to do so, and is increasingly cut off from international semiconductor supply chains.
Political goals: The Chinese Communist Party (CCP) is above all concerned with ensuring social stability and staying in power. It will not tolerate large language models that cannot reliably be controlled and prevented from leaking sensitive information, raising politically sensitive topics, or challenging the regime. Reliably prohibiting LLMs from doing any of these things is an unsolved technical problem. This has and will continue to stifle the development and deployment of Chinese LLMs for the Chinese market, as a single political faux pas will spell doom for the companies involved.

Therefore, as it currently stands, there is no AGI race between China and the US.

China Cannot Compete with the US to Reach AGI First Via Scaling LLMs

Today, training powerful LLMs like GPT-4 requires massive GPU clusters. And training LLMs on a larger number of GPUs is the most common and reliable way of increasing their capabilities. This is known as the bitter lesson in machine learning; increasing the compute a model is trained on leads to higher progress in the model's capabilities than any changes in the algorithm's architecture. So if China wants to “win the AGI race” against the US, it would need to maximise...

Posts

Wikitag Contributions

Comments