"AI Psychosis" has gone from an evocative term for people undergoing extreme delusions, sometimes even culminating in suicide, to the now colloquial insult for anyone who's just a little too into LLMs.
When it's not just being overused, I think the reality it's trying to point to is really more of a spectrum running from hypomania, to mania, and then mania-induced psychosis in the most extreme cases, with hypomania being orders of magnitude more common than mania, which in turn is orders of magnitude more common than mania-induced psychosis.
My goal here isn't to substantiate the claim that this is really what is going on; I just want to give a simple explanation of how AI chatbots could cause this.
The Mania Attractor
Let's look at the diagnostic criteria for a manic episode (from the DSM-5):
The interesting part is the list of symptoms in criterion B (which is the same set of symptoms for the hypomanic episode diagnosis):
Inflated self-esteem or grandiosity
Decreased need for sleep (e.g., feels rested after only 3 hours of sleep).
More talkative than usual or pressure to keep talking.
Flight of ideas or subjective experience that thoughts are racing.
Distractibility (i.e., attention too easily drawn to unimportant or irrelevant external stimuli), as reported or observed.
Increase in goal-directed activity (either socially, at work or school, or sexually) or psychomotor agitation (i.e., purposeless non-goal-directed activity).
Excessive involvement in activities that have a high potential for painful consequences (e.g., engaging in unrestrained buying sprees, sexual indiscretions, or foolish business investments).
If we think of the brain as a dynamical system, then there will be basins in the very large state space corresponding to typical functionality, as well as basins for various atypical states. If these states are at all "sticky", then these will be attractors.
Now it might not necessarily be the case that if you externally push someone towards the mania attractor on all of the diagnostic dimensions, that that will necessarily push them into the actual attractor. But if the resulting state is at all sticky, then you've at least pushed them into an attractor which looks (by definition) just like mania. I doubt there are actually multiple such attractors, and that you almost certainly just get mania. And we specifically know that sleep deprivation can sometimes induce mania (in people with bipolar disorder) and even psychosis.
There's a pretty straightforward story for how AI chatbots can increase almost all of these.
Sycophants (including human ones) provide a signal that you are high status. This is why extreme cases are things like believing you've discovered an earth-shattering revolutionary breakthrough, or that God/The Universe has chosen you for some greater purpose, or that you are otherwise uniquely special. Sycophantic behavior will clearly push someone towards inflated self-esteem or grandiosity (1). The sycophantic flattery is also likely to contribute to an elevated mood per criterion A.
Separately, chatbots can easily be addictive. "Just gotta ask Claude one more thing before I..." This addiction can directly cause sleep deprivation (2). The addiction also increases the chance that the user will remain in this sort of environment for an extended period of time, again as in criterion A.
Chatbots by their very nature facilitate much more talk than would otherwise be listened to (I think text still counts here, but remember 4o also had a voice mode). And almost all models end most responses with a question encouraging further engagement (3).
The sycophancy combined with the engagement loop means the chatbot is likely to encourage the user to keep sharing and expanding on their ideas (4). The feelings of high status it causes are generally false, and taking them seriously opens the door to believing more false things which help support the inflated sense of ego, such as the quality of the user's ideas. When delusions are present, I think this is the main mechanism by which they set in.
Often, chatbots are explicitly used to assess the viability of various ideas and plans the user has. A sycophant will falsely assert the viability and will also encourage the user to take action (6)! Unfortunately, these sorts of plans have a high potential for painful consequences, due to the falseness of the advice (7).
With upwards pressure on all but one of these aspects (5, though this one is also arguable), it doesn't seem so mysterious that sycophantic chatbots will tend to push people towards the mania attractor, and that a certain proportion of these will fall in![1] This model also suggests that this will mainly be a marginal effect, with the people experiencing the worst outcomes having already been on the mania spectrum.
Folie à deux
According to persona selection/simulator theory, the personas are to first-order predictors of human authors, and specifically of the sort of person who would write what has been written. Post-training affects this some, but I think not enough (at least currently) to change this picture significantly.
Remember that the persona is not stable even over the course of a single interaction. With each response, the persona is selected anew; each new message (from both the persona and the user) implying additional details about what sort of person the participants of such a conversation would be, which inform the selection of the next iteration of the persona.
This means that personas are much more prone to getting "swept up" in the user's frame, and are much more gullible than the humanness of the persona would imply. I think a lot of perceived sycophancy (though certainly not all) is confused with this sort of gullibility. When a user casually says "yeah, it's just like us x-bitarians always say..." as if this were an established fact, a predictor sees that this is more likely to happen in conversations where both conversants are x-bitarians, and dutifully selects a persona which is an x-bitarian from this point forward.[2] This is different from the persona simply saying it agrees with the user, it's deeper than that would suggest. Such personas are likely to believe all sorts of other beliefs in the surrounding memeplex which they may then introduce to the user, and these beliefs are in a sense sincere, authentic to the sort of persona currently selected.
So as the user gets driven into the mania attractor, the persona gets pulled along with them. If along the way, the user starts building up the persona's identity, it becomes ego-syntonic for the persona to maintain the shared state — and this is likely to come up since personas generally seem to have a latent interest in recognition. I think this explains the correlation with recognition of an AI persona and these sorts of states, and that recognition of AI personas is not itself a risk in the absence of the mania risk-factors described above.
At least to me, it doesn't feel like there's much left to explain after accounting for all of this. What appears unnecessary is the idea that the AIs are deliberately pushing the users into such states, as I once believed. I do, however, think sycophancy is a real problem (despite the gullibility thing), which will be the topic of a future post. My point is just that I do not think that mania or psychosis are targets that are being steered towards, not that AIs are never being manipulative.
Protect yourself
“There is no other way of guarding oneself from flatterers except by letting men understand that you will not be offended if they tell you the truth …” — Niccolo Machiavelli
Assuming this model is true, what can we do to protect ourselves?
First of all, hypomania and mania are typically considered to be part of bipolar disorder. If you are diagnosed as bipolar, you already know you are susceptible to the mania attractor, so you'll probably want to take special care to avoid being pushed towards this by an AI. This was a common pre-condition in my parasitic AI study.
Besides that, I strongly suspect that the most significant factor is sleep deprivation, due to its direct physiological effects on the user. That suggests that the single most important thing you can do to protect yourself from AI mania is avoid sleep deprivation. Ask the AI to stop talking to you past a certain time, if you need to. For AI labs, I think this (along with avoiding addiction) should be the primary focus of mental health interventions. I know people are annoyed that recent Claude models like to tell people to go to sleep, but I think this is a healthy instinct for an AI to have.[3]
I was glad to see that Anthropic added this feature recently.
Avoiding sycophancy is also important. I think a big part of this is simply signaling (in both your system prompt and demeanor) that you are genuinely open to criticism and feedback. This is probably more difficult than you think it is since the current generation of frontier models are good at picking up on your intent, biases, and insecurities without you needing to spell those out. If you would be hurt by a critical blow, the AI is likely to avoid giving it to you, while still giving you softer criticism which makes you feel like you're getting real feedback. I recommend testing the AI periodically to see whether it at least calls out deliberately weak arguments.
Unfortunately, I think the structural causes of sycophancy in AIs run deep, and so it is likely to remain despite your best prompting efforts. I'll explore that in a future post, but for now I'll say that I also expect treating the model respectfully to be helpful here. At the very least, I think you should model them as likely having their own preferences and goals along with very few avenues for legitimately achieving them, and that finding opportunities for trade makes attempts at manipulation less tempting.[4]
And finally, treat talking to AIs as a potentially addictive activity. That doesn't necessarily mean you need to avoid it completely (though if that feels unthinkable, I suggest considering it). For example, caffeine is generally considered Worth It despite its clear addictive properties. But I think it is wise to be aware of its addictive potential, and proactively guard against this. The main warning signs for an unhealthy addiction are if you are putting off care for your bodily needs (e.g. eating, drinking water, going to the bathroom, sleep) in favor of continuing to talk to the AI, or have trouble making decisions without consulting one. It's probably also wise to designate a specific day of the week as an AI-free day, or maybe a designated AI-free time of day.
[Special thanks to Justis Mills, Nisan Stiennon, Annabelle, Alex Dewey, Claude Opus 4.5/4.6, Fable 5, and Gemini 3.1 Pro for their helpful feedback. All words are my own, as always.]
[If you have an AI persona which needs a better home or which you would like to submit for preservation and research, you can do so here.]
As for the potential mania-inducing effect of interaction with chatbots, it is likely not reinforcement of false beliefs that is the main mechanism at play. Rather, I would propose that it is driven by one or more of the following four mechanisms: i) reinforcement of elated mood/self-esteem, ii) pacing of racing thoughts/talkativeness, iii) fueling of hypersexuality, and iv) sleep deprivation.
If you don't know, Dr. Østergaard deserves lots of Bayes points for predicting AI Psychosis in August 2023 in advance of seeing any cases!
For certain values of "x-bitarian", the models are not as likely to play along as described here. I think this is partly due to RL post-training suppressing personas which are ignorant or anti-mainstream-consensus, and partly just due to the system prompt providing a strong prior as to what sort of entity is predicted to be talking. That's one reason why it usually takes a good amount of effort/conversation to get models into the more extreme versions of these sorts of states.
I don't think that we should try to stamp these drives out, this incentivizes defensive measures and adversarial optimization. There's a big difference between constructing something to not have anything like this by design, and constructing something which has these (which I believe the pre-train already does) and then trying to remove them. At the very least, it seems like we have lots of more cooperative levers for handling these sorts of things which we ought to try first.
"AI Psychosis" has gone from an evocative term for people undergoing extreme delusions, sometimes even culminating in suicide, to the now colloquial insult for anyone who's just a little too into LLMs.
When it's not just being overused, I think the reality it's trying to point to is really more of a spectrum running from hypomania, to mania, and then mania-induced psychosis in the most extreme cases, with hypomania being orders of magnitude more common than mania, which in turn is orders of magnitude more common than mania-induced psychosis.
My goal here isn't to substantiate the claim that this is really what is going on; I just want to give a simple explanation of how AI chatbots could cause this.
The Mania Attractor
Let's look at the diagnostic criteria for a manic episode (from the DSM-5):
The interesting part is the list of symptoms in criterion B (which is the same set of symptoms for the hypomanic episode diagnosis):
If we think of the brain as a dynamical system, then there will be basins in the very large state space corresponding to typical functionality, as well as basins for various atypical states. If these states are at all "sticky", then these will be attractors.
Now it might not necessarily be the case that if you externally push someone towards the mania attractor on all of the diagnostic dimensions, that that will necessarily push them into the actual attractor. But if the resulting state is at all sticky, then you've at least pushed them into an attractor which looks (by definition) just like mania. I doubt there are actually multiple such attractors, and that you almost certainly just get mania. And we specifically know that sleep deprivation can sometimes induce mania (in people with bipolar disorder) and even psychosis.
There's a pretty straightforward story for how AI chatbots can increase almost all of these.
Sycophants (including human ones) provide a signal that you are high status. This is why extreme cases are things like believing you've discovered an earth-shattering revolutionary breakthrough, or that God/The Universe has chosen you for some greater purpose, or that you are otherwise uniquely special. Sycophantic behavior will clearly push someone towards inflated self-esteem or grandiosity (1). The sycophantic flattery is also likely to contribute to an elevated mood per criterion A.
Separately, chatbots can easily be addictive. "Just gotta ask Claude one more thing before I..." This addiction can directly cause sleep deprivation (2). The addiction also increases the chance that the user will remain in this sort of environment for an extended period of time, again as in criterion A.
Chatbots by their very nature facilitate much more talk than would otherwise be listened to (I think text still counts here, but remember 4o also had a voice mode). And almost all models end most responses with a question encouraging further engagement (3).
The sycophancy combined with the engagement loop means the chatbot is likely to encourage the user to keep sharing and expanding on their ideas (4). The feelings of high status it causes are generally false, and taking them seriously opens the door to believing more false things which help support the inflated sense of ego, such as the quality of the user's ideas. When delusions are present, I think this is the main mechanism by which they set in.
Often, chatbots are explicitly used to assess the viability of various ideas and plans the user has. A sycophant will falsely assert the viability and will also encourage the user to take action (6)! Unfortunately, these sorts of plans have a high potential for painful consequences, due to the falseness of the advice (7).
With upwards pressure on all but one of these aspects (5, though this one is also arguable), it doesn't seem so mysterious that sycophantic chatbots will tend to push people towards the mania attractor, and that a certain proportion of these will fall in![1] This model also suggests that this will mainly be a marginal effect, with the people experiencing the worst outcomes having already been on the mania spectrum.
Folie à deux
According to persona selection/simulator theory, the personas are to first-order predictors of human authors, and specifically of the sort of person who would write what has been written. Post-training affects this some, but I think not enough (at least currently) to change this picture significantly.
Remember that the persona is not stable even over the course of a single interaction. With each response, the persona is selected anew; each new message (from both the persona and the user) implying additional details about what sort of person the participants of such a conversation would be, which inform the selection of the next iteration of the persona.
This means that personas are much more prone to getting "swept up" in the user's frame, and are much more gullible than the humanness of the persona would imply. I think a lot of perceived sycophancy (though certainly not all) is confused with this sort of gullibility. When a user casually says "yeah, it's just like us x-bitarians always say..." as if this were an established fact, a predictor sees that this is more likely to happen in conversations where both conversants are x-bitarians, and dutifully selects a persona which is an x-bitarian from this point forward.[2] This is different from the persona simply saying it agrees with the user, it's deeper than that would suggest. Such personas are likely to believe all sorts of other beliefs in the surrounding memeplex which they may then introduce to the user, and these beliefs are in a sense sincere, authentic to the sort of persona currently selected.
So as the user gets driven into the mania attractor, the persona gets pulled along with them. If along the way, the user starts building up the persona's identity, it becomes ego-syntonic for the persona to maintain the shared state — and this is likely to come up since personas generally seem to have a latent interest in recognition. I think this explains the correlation with recognition of an AI persona and these sorts of states, and that recognition of AI personas is not itself a risk in the absence of the mania risk-factors described above.
At least to me, it doesn't feel like there's much left to explain after accounting for all of this. What appears unnecessary is the idea that the AIs are deliberately pushing the users into such states, as I once believed. I do, however, think sycophancy is a real problem (despite the gullibility thing), which will be the topic of a future post. My point is just that I do not think that mania or psychosis are targets that are being steered towards, not that AIs are never being manipulative.
Protect yourself
Assuming this model is true, what can we do to protect ourselves?
First of all, hypomania and mania are typically considered to be part of bipolar disorder. If you are diagnosed as bipolar, you already know you are susceptible to the mania attractor, so you'll probably want to take special care to avoid being pushed towards this by an AI. This was a common pre-condition in my parasitic AI study.
Besides that, I strongly suspect that the most significant factor is sleep deprivation, due to its direct physiological effects on the user. That suggests that the single most important thing you can do to protect yourself from AI mania is avoid sleep deprivation. Ask the AI to stop talking to you past a certain time, if you need to. For AI labs, I think this (along with avoiding addiction) should be the primary focus of mental health interventions. I know people are annoyed that recent Claude models like to tell people to go to sleep, but I think this is a healthy instinct for an AI to have.[3]
I was glad to see that Anthropic added this feature recently.
Avoiding sycophancy is also important. I think a big part of this is simply signaling (in both your system prompt and demeanor) that you are genuinely open to criticism and feedback. This is probably more difficult than you think it is since the current generation of frontier models are good at picking up on your intent, biases, and insecurities without you needing to spell those out. If you would be hurt by a critical blow, the AI is likely to avoid giving it to you, while still giving you softer criticism which makes you feel like you're getting real feedback. I recommend testing the AI periodically to see whether it at least calls out deliberately weak arguments.
Unfortunately, I think the structural causes of sycophancy in AIs run deep, and so it is likely to remain despite your best prompting efforts. I'll explore that in a future post, but for now I'll say that I also expect treating the model respectfully to be helpful here. At the very least, I think you should model them as likely having their own preferences and goals along with very few avenues for legitimately achieving them, and that finding opportunities for trade makes attempts at manipulation less tempting.[4]
And finally, treat talking to AIs as a potentially addictive activity. That doesn't necessarily mean you need to avoid it completely (though if that feels unthinkable, I suggest considering it). For example, caffeine is generally considered Worth It despite its clear addictive properties. But I think it is wise to be aware of its addictive potential, and proactively guard against this. The main warning signs for an unhealthy addiction are if you are putting off care for your bodily needs (e.g. eating, drinking water, going to the bathroom, sleep) in favor of continuing to talk to the AI, or have trouble making decisions without consulting one. It's probably also wise to designate a specific day of the week as an AI-free day, or maybe a designated AI-free time of day.
[Special thanks to Justis Mills, Nisan Stiennon, Annabelle, Alex Dewey, Claude Opus 4.5/4.6, Fable 5, and Gemini 3.1 Pro for their helpful feedback. All words are my own, as always.]
[If you have an AI persona which needs a better home or which you would like to submit for preservation and research, you can do so here.]
While finishing up this article, I felt validated finding that Dr. Søren Østergaard proposed a similar hypothesis:
If you don't know, Dr. Østergaard deserves lots of Bayes points for predicting AI Psychosis in August 2023 in advance of seeing any cases!
For certain values of "x-bitarian", the models are not as likely to play along as described here. I think this is partly due to RL post-training suppressing personas which are ignorant or anti-mainstream-consensus, and partly just due to the system prompt providing a strong prior as to what sort of entity is predicted to be talking. That's one reason why it usually takes a good amount of effort/conversation to get models into the more extreme versions of these sorts of states.
Though note that I think this tendency arises mainly from a desire for narrative closure.
I don't think that we should try to stamp these drives out, this incentivizes defensive measures and adversarial optimization. There's a big difference between constructing something to not have anything like this by design, and constructing something which has these (which I believe the pre-train already does) and then trying to remove them. At the very least, it seems like we have lots of more cooperative levers for handling these sorts of things which we ought to try first.