Epistemic status: I have read the recent major debates.[1][2][3][4]I am not taking a strong position of doing nothing, or accelerating, or stopping immediately; rather, I want to try to look at this with a more structural attitude.
My view is that these recent major debates are all highly focused on a single point: the moment when the catastrophe finally happens. I call this mindset a one-way-time mindset. The structure of this mindset is this: the coming of a future doomsday is believed to sweep away everything, which then makes everything else seem not very important compared with it. This belief itself changes the weight given to things.
I want to make two points. First, I want to explicitly rebut a claim some people make: that we need to stop before things end.[2][4]My rebuttal is: anything is easiest to stop before it has started; as things become more and more certain, the thing accelerates more and more and becomes harder and harder to stop. For example, when things like neural networks were still uncertain, only a few people with conviction were trying, but even they were not completely sure. Then, after a series of key nodes, such as AlexNet, the Transformer, and ChatGPT, things have become more and more validated and more and more certain. So if the doomsday node, or rather the superintelligence node, really exists, then the closer we get to it, the more humanity, because of all kinds of interests or fundamentally different values, will only go faster and faster and become harder and harder to stop. What you can block is only one path, but there will be countless paths all leading to that place. As for key nodes, we have in fact already crossed many of them, like the ones I just mentioned, and each key node is already irreversible as far as technology diffusion is concerned; our lives have already been changed.
Second, this single-point view makes people feel that a few key nodes and a few figures can stop it. This mindset, I feel, gives people a sense of control: because if a small number of nodes can reverse or pause this thing, then it seems there is still a definite direction for effort, and so, following that one-way-time mindset, one can align one's life toward that direction, to strive and to try to grasp something. This in itself comes with a yes-or-no mindset: the priority of this thing is placed ahead of the priority of everything else. This is inherently a mutually reinforcing process. This mindset itself actually leads to the microdose that Kulveit talks about.[1]Except that this dose is caused not only inside companies, but also by this whole single-point-doomsday worldview.
Through these points, I want to oppose this AI-doomsday-oriented, all-or-nothing mode of thinking, and to shift the focus to irreversible technology diffusion.[5] As for getting so-and-so to stop, I have no hope. I think we should, to some extent, move from the one-way-time parallel world inside the microdose into the social and political life that is already happening, to make a comprehensive response to this new kind of thing that has never existed before, rather than just patching things up, for example by collecting some taxes or handing out some subsidies, and so on.[6][7]
Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, David Duvenaud, "Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development", arXiv:2501.16946, 2025. https://arxiv.org/abs/2501.16946
I couldn't follow this; it's quite abstract. Using some examples might clarify your intent. You seem to be saying we shouldn't try to prevent doom? That position is counterintuitive to say the least and could use more clear and careful argumentation.
Thanks. My post was written about 2 weeks ago, as a direct response to the earlier debates. The "microdosing" framing borrows the rhetoric of that post, but I didn't clearly explain this usage or my intent, so it is indeed a bit confusing.
What I want to explain is that where we differ is the definition of doom. I absolutely think we should prevent doom, but doom is more likely to come from power asymmetry, which in some cases is already dismantling our society's guardrails. The catastrophic doomsday scenario is clearly the worst case. But even if that scenario were guaranteed to be prevented, the other kind of doom has most likely already happened. So I think we should distribute our attention more evenly across the different kinds of doom. I’ll try to follow up with some examples later.
Epistemic status: I have read the recent major debates.[1][2][3][4]I am not taking a strong position of doing nothing, or accelerating, or stopping immediately; rather, I want to try to look at this with a more structural attitude.
My view is that these recent major debates are all highly focused on a single point: the moment when the catastrophe finally happens. I call this mindset a one-way-time mindset. The structure of this mindset is this: the coming of a future doomsday is believed to sweep away everything, which then makes everything else seem not very important compared with it. This belief itself changes the weight given to things.
I want to make two points. First, I want to explicitly rebut a claim some people make: that we need to stop before things end.[2][4]My rebuttal is: anything is easiest to stop before it has started; as things become more and more certain, the thing accelerates more and more and becomes harder and harder to stop. For example, when things like neural networks were still uncertain, only a few people with conviction were trying, but even they were not completely sure. Then, after a series of key nodes, such as AlexNet, the Transformer, and ChatGPT, things have become more and more validated and more and more certain. So if the doomsday node, or rather the superintelligence node, really exists, then the closer we get to it, the more humanity, because of all kinds of interests or fundamentally different values, will only go faster and faster and become harder and harder to stop. What you can block is only one path, but there will be countless paths all leading to that place. As for key nodes, we have in fact already crossed many of them, like the ones I just mentioned, and each key node is already irreversible as far as technology diffusion is concerned; our lives have already been changed.
Second, this single-point view makes people feel that a few key nodes and a few figures can stop it. This mindset, I feel, gives people a sense of control: because if a small number of nodes can reverse or pause this thing, then it seems there is still a definite direction for effort, and so, following that one-way-time mindset, one can align one's life toward that direction, to strive and to try to grasp something. This in itself comes with a yes-or-no mindset: the priority of this thing is placed ahead of the priority of everything else. This is inherently a mutually reinforcing process. This mindset itself actually leads to the microdose that Kulveit talks about.[1]Except that this dose is caused not only inside companies, but also by this whole single-point-doomsday worldview.
Through these points, I want to oppose this AI-doomsday-oriented, all-or-nothing mode of thinking, and to shift the focus to irreversible technology diffusion.[5] As for getting so-and-so to stop, I have no hope. I think we should, to some extent, move from the one-way-time parallel world inside the microdose into the social and political life that is already happening, to make a comprehensive response to this new kind of thing that has never existed before, rather than just patching things up, for example by collecting some taxes or handing out some subsidies, and so on.[6][7]
Jan Kulveit, "Microdosing the spirit of Anthropic", LessWrong, 2026-10-30. https://www.lesswrong.com/posts/oL7aFoRmRXGwEMhCm/microdosing-the-spirit-of-anthropic
Scott Alexander, "Against SURA", LessWrong, 2026-10-31. https://www.lesswrong.com/posts/xikGZ8fuhdG6DApAo/against-sura
Zvi Mowshowitz, "Scott Alexander's Against SURA Seems Deeply Confused", LessWrong, 2026-11-02. https://www.lesswrong.com/posts/ziMP83D2JYWRQ5jtu/scott-alexander-s-against-sura-seems-deeply-confused
Geoffrey Irving, "Against strawmanning unilateral pauses", LessWrong, 2026-11-03. https://www.lesswrong.com/posts/enzpS72uDmomiWN7a/against-strawmanning-unilateral-pauses
Daron Acemoglu, Introduction to Modern Economic Growth, Princeton University Press, 2009.
Wenjun Cao, "Human Finitude: AI and the Foundation of Civilization", working paper, 2026. https://directedlink.github.io/the-order-of-finite-beings/
Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, David Duvenaud, "Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development", arXiv:2501.16946, 2025. https://arxiv.org/abs/2501.16946