I'm interested in understanding why people sometimes start acting strange after extended conversations with language models. Davidad's recent shift in direction is the example that comes to mind, as well as accounts of AI psychosis. I think that one of the more naive ways to investigate this would be to measure the R0 (reproduction number) of attractor states like spiritual bliss and spiralism in a multi-agent setup. Are there factors that make people more or less susceptible to this effect? I believe that model personas are a monoculture, so prompt injection attacks which work on one GPT instance could more easily transfer across to another GPT instance than it could to a human, so I'm not very worried about humanity being defeated by mind hacking brainworms summoned by slop quines invading the noosphere, but for public health I would recommend limiting LLM chat sessions to not longer than 30-90 minutes. Products such as Claude Code Web default to using an isolated sandbox container instead of allowing application system calls to the kernel, yet they directly expose users to model outputs, which might have subtle persuasive effects that are not yet well understood (isn't it strange how many safety researchers talk to Opus for a while and come out much more optimistic about the future, thinking we'll get alignment by default?). I think that research into isolating and constraining model output tokens is a direction with a straightforward pathway to product integration in frontier systems which are likely to have stronger containment strategies in the future. However since this area deals with human subjects it doesn't seem to be an especially good fit for independent research and would likely be better suited for trusted institutions with a strong and verifiable commitment to ethical standards.
why people sometimes start acting strange after extended conversations with language models
My guess it that many people are in a half-crazy state, where their minds are already drawn towards insanity, but the everyday feedback they keep getting from other humans keeps them sane.
Now replace the human feedback with AI sycophancy, and instead of pushback they start getting what feels like social permission to go fully insane.
It is similar to a cult, where a group of people push each other towards unhealthy behaviors. But with an AI, it's like a cult consisting of you and your mirror image that has artificially boosted INT but not WIS.