China is weak in frontier AI safety, and the domestic field is currently facing significant challenges and headwinds. Why? Here's a TL;DR field note after talking with various researchers on the Chinese side.
[Obligatory disclaimer: This post is written in my personal capacity and is not intended to represent the official stance of any organization(s) that I am in or have been in contact with]
Chinese AI startups and hyperscaler labs alike have very little frontier risk situational awareness. People can maybe name cybersecurity, but not much more than that. There is no intimate understanding of frontier risks. It's generally not deliberately ignorant or strategically not pursuing them; it's a complete vacuum of imagination. The Western field had FOOM debates in 2011, while China has nothing analogous in the mainstream AI research field consciousness before 2023.
Getting any funding at all is hard. If you are a domestic organization, being a non-profit means heavy suspicion (this had already played out in one of the most prominent Chinese safety orgs). Getting foreign EA funding/grants is basically a non-starter, given China's stringent foreign non-profit rules. Domestically, there’s very little local EA money both in absolute terms and in terms of percentage of total non-profit money.
Therefore, many safety-pilled people are forced to do for-profits, selling safety & observability products or courses on frontier safety to domestic lab employees, while using most of the profits to conduct basically non-profit safety research. I think this is such an underdiscussed aspect of the Chinese ecosystem, and I'm afraid the Western sphere may misunderstand the phenomenon as "this is just ultimately capabilities research or working for the state's goals".
Getting frontier AI assistance is difficult and costly. There are no "Claude for Chinese safety researchers" carve-outs; in fact, they ban accounts regularly even for accounts conducting clear civilian AI safety work (if anyone cared to review the transcripts), just because of the simple fact that they detect that the user might be from China by region. You still need the best models to assist with research anyways, so this measure realistically incurs disproportionately high costs on researchers with less funding and resources, discouraging them early.
Institutional foreign outreach is structurally risky. There is a strong fear of reaching out to METR, Apollo, Redwood, etc. because it's "涉外" (meaning "involving a foreign entity"). Everyone is trying to dance on a tightrope. The most feasible path I see is for individual US researchers to contact Chinese safety-pilled organizations in an individual capacity and engage in earnest collaboration.
Much of this is very daunting. I think some remediations could be viable and immediately beneficial, such as giving subscriptions and compute in a supervised, monitored capacity to individuals working in the civilian frontier safety field. There are various things that I can't gauge the will of the labs or the organizations to do by pure guesswork, so I'd like to hear people step up and discuss.
China is weak in frontier AI safety, and the domestic field is currently facing significant challenges and headwinds. Why? Here's a TL;DR field note after talking with various researchers on the Chinese side.
[Obligatory disclaimer: This post is written in my personal capacity and is not intended to represent the official stance of any organization(s) that I am in or have been in contact with]
Much of this is very daunting. I think some remediations could be viable and immediately beneficial, such as giving subscriptions and compute in a supervised, monitored capacity to individuals working in the civilian frontier safety field. There are various things that I can't gauge the will of the labs or the organizations to do by pure guesswork, so I'd like to hear people step up and discuss.