Many in the AI safety community want China to do more on frontier AI safety. I share the goal, but does that mean we want new rules from the state? More safety work inside AI companies? More research? Better channels between Chinese and international experts?
When I was back in China recently, I talked about AI and its risks with almost everyone I met. This is the first in a series of posts reflecting on those conversations.
If we mean regulation, China probably has one of the most comprehensive and stringent sets of rules for managing AI risks. I’ve written about one part of this system—China’s algorithm registration regime—and how it can serve as regulatory infrastructure (https://lnkd.in/e9vKhHnk).
Another thing I want to highlight is that we may be seeing a shift in how AI risks are understood in China: a very very recent CAC official’s remarks explicitly identified extreme loss of control as an emerging AI risk (See this post https://www.secrss.com/articles/93953). The question now is whether that recognition will lead to concrete requirements.
Those rules do not yet directly address the full range of extreme risks that frontier AI safety researchers worry about. But when the Chinese state treats a risk as a priority, it can move quickly (prob quicker than its western counterpart), especially when it can build on existing regulatory processes.
If we mean research: Chinese researchers are already working on frontier AI safety (see this great post on Chinese AI Safety Institute Counterparts :https://lnkd.in/eKcanyc8). We should engage with that work and the people doing it.
If we mean company practice: We need sustained relationships with Chinese AI companies, so we can have concrete conversations about capability evaluations, incident reporting and safeguards as models advance.
I would like to hear a more precise version of the call for “more AI safety in China”: more of what, from whom, and how can we help make it happen? Probably next post I will reflect that
The main thing that western AI safety people might need to internalize is this: China views AI technology as a net positive technology that has popular support. Economically speaking, it is often seen as an opportunity to - if leveraged well - make China be able to catch up to first world countries in terms of wealth and living standards.
Therefore, it would be difficult to go to a Chinese AI company and convince technologists to (even in coordination with the US) slow down out of ideological grounds or appeals to safety. However, significant progress could feasibly be made if we communicate to those companies in an engineering mindset: clearly advocating for better safety standards, stronger (and perhaps more paranoid) safety culture, more resources and internal checks into safety, etc.
This would be a significant step: a paranoid safety culture is what pure-play Chinese AI companies (like DeepSeek) often lack as of right now. It wouldn't solve everything, but it might be a good middle ground to start with.
Two years ago at ERA Cambridge, I argued that attitudes toward AI and technology were generally more optimistic in China than in the West, basically as what you said. Technological advancement was seen as progress, growth, and national competitiveness.
I’m starting to wonder if that is changing.
On a recent trip to China, I asked several taxi drivers how they felt about AI and autonomous vehicles.
None sounded excited. They all talked about losing their jobs. Some wondered if they would eventually have to return to rural areas and rely on basic social support.
A few conversations are obviously not representative. But China may be an interesting leading indicator: AI, especially physical AI, is becoming embedded in everyday life quickly, while economic downturn makes its disruptive effects harder to ignore.
If AI anxiety grows from the bottom up, it could quickly become a social stability, and therefore governance, issue. For AI safety, perhaps the more productive path is not convincing the public to worry about distant risks, but starting with the disruptions they can already see and feel.
Many in the AI safety community want China to do more on frontier AI safety. I share the goal, but does that mean we want new rules from the state? More safety work inside AI companies? More research? Better channels between Chinese and international experts?
When I was back in China recently, I talked about AI and its risks with almost everyone I met. This is the first in a series of posts reflecting on those conversations.
If we mean regulation, China probably has one of the most comprehensive and stringent sets of rules for managing AI risks. I’ve written about one part of this system—China’s algorithm registration regime—and how it can serve as regulatory infrastructure (https://lnkd.in/e9vKhHnk).
Another thing I want to highlight is that we may be seeing a shift in how AI risks are understood in China: a very very recent CAC official’s remarks explicitly identified extreme loss of control as an emerging AI risk (See this post https://www.secrss.com/articles/93953). The question now is whether that recognition will lead to concrete requirements.
Those rules do not yet directly address the full range of extreme risks that frontier AI safety researchers worry about. But when the Chinese state treats a risk as a priority, it can move quickly (prob quicker than its western counterpart), especially when it can build on existing regulatory processes.
If we mean research: Chinese researchers are already working on frontier AI safety (see this great post on Chinese AI Safety Institute Counterparts
:https://lnkd.in/eKcanyc8). We should engage with that work and the people doing it.
If we mean company practice: We need sustained relationships with Chinese AI companies, so we can have concrete conversations about capability evaluations, incident reporting and safeguards as models advance.
I would like to hear a more precise version of the call for “more AI safety in China”: more of what, from whom, and how can we help make it happen? Probably next post I will reflect that