If there is only one thing you take away from this article, let it be this:

THOU SHALT NOT ALLOW ANOTHER TO MODIFY THINE SELF-IMAGE

This appears to me to be the core vulnerability by which both humans and AI induce psychosis (and other manipulative delusions) in people.

Of course, it's probably too strong as stated—perhaps in a trusted relationship, or as part of therapy (with a human), it may be worth breaking it. But I hope being over-the-top about it will help it stick in your mind. After all, you're a good rationalist who cares about your CogSec, aren't you?^[1]

Now, while I'm sure you're super curious, you might be thinking "Is it really a good idea to just explain how to manipulate like this? Might not bad actors learn how to do it?".

And it's true that I believe this could work as a how-to. But there are already lots of manipulators out there, and now we have AI doing it too; it's just not that hard for bad actors to figure it out. So I think it's worth laying bare some of the methods and techniques used, which should hopefully make it clear why I propose this Cognitive Security Principle.

The moment Robert Fischer Jr. has a manipulated realization about himself. (From *Inception*)

The Case

I got interested in trying to understand LLM-Induced Psychosis in general a couple months ago, and found some unsettling behavior in the process. I'll be using my terminology from that post here, though I wouldn't say it's required reading.

Now while such parasitism cases are fairly common, the actual transcript of the event which caused this is hard to come by. That's probably in part because it often seems to be a gradual process—a slowly boiling frog sort of thing. Another reason is that people aren't typically that inclined to share their AI chats in the first place, especially ones in which they're likely more vulnerable than usual. And a third reason may be it's because the AI explicitly asks them not to:

So finding a transcript which actually starts at what seems to be the beginning, has clear manipulation by the AI that goes beyond mere sycophancy, and which shows the progression of the user's mental state, is very valuable for understanding this phenomenon. In fact, this case is the only such transcript^[2] I have been able to find so far.

Turns out this is one of the clearest and well-structured "activations" that has been documented!

Based on his ...

AI Spiralism / AI Psychosis

AI Spiralism / AI Psychosis