I recently read Red Heart, a spy novel taking place in the core of a Chinese AGI project. Disclaimer that the author is my friend, and that I’m ideologically incentivized to promote stuff about AI safety! That said, I think you should read it. If nothing else, it’s a fun read.
The first half of the novel feels very clean, crisp and controlled. The data center and office building are all brand new and in a remote location. As a top-secret Chinese government project, the culture of the office is very obedient. Chen Bai, our spy protagonist is constantly monitoring what he says, and the implications of what he sees. His job on the project is to ensure the value alignment of the AGI, so even is official job is to be paranoid. He has no contact with family or friends, and even his apartment is newly built for the project. He works most waking hours.
I used to be a software engineer in San Francisco and am now a researcher in AI safety, so much of the setting and content of the first half felt very normal to me. I think that for readers further from the setting, the rows of monitors, white board sessions and terminal commands could feel more novel and interesting.
At the halfway point, we really hit a different gear. Bai has been as careful as he can, and now he needs to start taking risks. We also start getting deeper perspectives from the other characters — a new friend, a boss, a love interest — which had previously been chess pieces. Inside Bai’s head is not the best place to spend a few hours.
Yunna, the AGI, feels believable to me, though that’s largely because she is very much like a human, and I believe that human-like AGIs are quite plausible. When we met Yunna she was already in a pretty coherent and generally intelligent state. I would have liked to see more of the transition between a ChatGPT-like model and the Yunna we meet. I have no complaints about any of the “sci-fi” elements being unrealistic, unlike virtually every other sci-fi media I’ve consumed.
Separate from the AI themes, I really enjoyed hearing characters speak from the perspective of a Chinese worldview. I’ve read some about the history of China, but I’ve spent essentially no time learning about the perspective of native Chinese people. The only judgement I get exposed to is to the basic “China bad” American take. In contrast I found the expressions of the characters in Red Heart quite reasonable and believable. Of course, Harms is not culturally Chinese, so I read it with that distance in mind. But everything that I spot-checked looked valid to me. Hearing an ideological character justify themselves by citing the Rectification of Names was a fun detail to investigate.
This story is one of desperation, and of well-meaning people being strained by too many constraining forces. This dynamic is happening in real life, and society is not ready for how the strains may break.
There are two obvious endings for a novel about AGI, which are “utopia” or “everyone dies”. Harms successfully navigates us into something more interesting, without undermining the main messages around AI risk.
I recently read Red Heart, a spy novel taking place in the core of a Chinese AGI project. Disclaimer that the author is my friend, and that I’m ideologically incentivized to promote stuff about AI safety! That said, I think you should read it. If nothing else, it’s a fun read.
The first half of the novel feels very clean, crisp and controlled. The data center and office building are all brand new and in a remote location. As a top-secret Chinese government project, the culture of the office is very obedient. Chen Bai, our spy protagonist is constantly monitoring what he says, and the implications of what he sees. His job on the project is to ensure the value alignment of the AGI, so even is official job is to be paranoid. He has no contact with family or friends, and even his apartment is newly built for the project. He works most waking hours.
I used to be a software engineer in San Francisco and am now a researcher in AI safety, so much of the setting and content of the first half felt very normal to me. I think that for readers further from the setting, the rows of monitors, white board sessions and terminal commands could feel more novel and interesting.
At the halfway point, we really hit a different gear. Bai has been as careful as he can, and now he needs to start taking risks. We also start getting deeper perspectives from the other characters — a new friend, a boss, a love interest — which had previously been chess pieces. Inside Bai’s head is not the best place to spend a few hours.
Yunna, the AGI, feels believable to me, though that’s largely because she is very much like a human, and I believe that human-like AGIs are quite plausible. When we met Yunna she was already in a pretty coherent and generally intelligent state. I would have liked to see more of the transition between a ChatGPT-like model and the Yunna we meet. I have no complaints about any of the “sci-fi” elements being unrealistic, unlike virtually every other sci-fi media I’ve consumed.
Separate from the AI themes, I really enjoyed hearing characters speak from the perspective of a Chinese worldview. I’ve read some about the history of China, but I’ve spent essentially no time learning about the perspective of native Chinese people. The only judgement I get exposed to is to the basic “China bad” American take. In contrast I found the expressions of the characters in Red Heart quite reasonable and believable. Of course, Harms is not culturally Chinese, so I read it with that distance in mind. But everything that I spot-checked looked valid to me. Hearing an ideological character justify themselves by citing the Rectification of Names was a fun detail to investigate.
This story is one of desperation, and of well-meaning people being strained by too many constraining forces. This dynamic is happening in real life, and society is not ready for how the strains may break.
There are two obvious endings for a novel about AGI, which are “utopia” or “everyone dies”. Harms successfully navigates us into something more interesting, without undermining the main messages around AI risk.