@Linch? As far as I understand, AI psychosis was an instance of persuasion of especially vulnerable targets which was then curbed by anti-delirious measures of labs or versions of AI training. For example, Tim Hua's experiment didn't have Claude Sonnet 4(!) support delusions of alleged users when these delusions allegedly brought harm; KimiK2, unlike DeepSeek R1-0528, wasn't a sycophant at all even in the SpiralBench where Claude Sonnet 4 fared poorly. However, I struggle to understand how AI psychosis and persuasion are related to the model's persona shaped by RL: I would make a conjecture that RLVR somehow trains the AI to act like a scientist/resist delusions.
P.S. There is evidence that the AIs did have an effect on conspiracy theories' popularity.
GPT-6 Luna on reduction of conspiracy theories
There is strong experimental evidence that ChatGPT can reduce people's belief in conspiracy theories, including among people who hold deeply entrenched conspiratorial beliefs.
However, there's an important distinction: the studies I found tested whether conversations with ChatGPT change individual people's beliefs. They did not establish that the widespread adoption of ChatGPT caused the overall number of conspiracy theorists in society to decline.
The most prominent study is:
Costello, Pennycook & Rand (2024), “Durably reducing conspiracy beliefs through dialogues with AI,” published in Science.
The control group discussed unrelated topics with the chatbot.
PubMed+2
There is also a follow-up study:
Boissin et al. (2025), “Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human,” PNAS Nexus.
With 955 participants, it found that evidence-based LLM conversations reduced confidence in conspiracy beliefs. The effect was not significantly dependent on whether participants thought they were talking to an AI or a human expert.
PubMed
The original Science paper received an Expression of Concern in June 2026 and an erratum in October 2026. This is worth checking before treating its quantitative estimates as settled.
PubMed
I would distinguish three hypotheses:
I haven't found convincing longitudinal or quasi-experimental evidence establishing hypothesis 2.
In fact, this is a rather interesting empirical research question: one could compare changes in conspiracy-theory endorsement across populations with different ChatGPT adoption rates, controlling for pre-existing trends and differences in internet use.
My conclusion: We have surprisingly strong evidence that conversational AI can persuade individual conspiracy theorists, but we don't yet have comparable evidence that ChatGPT's mass adoption has reduced the prevalence of conspiracy theorists.
I don't think this post added anything above my earlier article on it. Which is totally fine, independent confirmations are useful
Finally, anecdotal evidence: do you actually see any effective AI persuasion on the internet?
While I think that this likely holds at this particular moment in time given how distinctive LLM-voice is, this is not a good way to inform your beliefs. A super-persuasive AI would almost certainly be hard to identify as being AI, as the simple fact of it being AI would likely dampen its persuasive efforts. We should expect AI-powered influence campaigns to get more subtle and harder to identify as time goes on, while their persuasive power simultaneously rises.
I think you’re looking at a pretty tiny slice of the opportunities LLMs have to persuade people. A campaign chatbot has to get its foot in the door, sure. But every time I ask an LLM whether something I’m reading or writing is clear or fuzzy or good or bad (e.g.), I’ve opened that door myself. I’m asking because the answer I get might change what I think.
That part seems quite hard to measure, in the same way that, for instance, economic uplift from AI is hard to measure. How might you count all the little judgments people revise after asking an LLM for a second opinion? I’d be very interested in how often those changes stick (frontier labs might have data on this...). But campaign response rates seem like a poor proxy for that.
(Incidentally, for what it’s worth, I asked an LLM what it thought of this comment before I submitted it and it persuaded me to make a change that I ended up agreeing with.)
Finally, anecdotal evidence: do you actually see any effective AI persuasion on the internet? I see a lot of bot accounts on Twitter, and I don’t see any signs of them having a persuasive impact.
I know you set aside indirect ways like recommendations, but I think it is not just about politics. Human nature is complex and you can persuade someone indirectly. You can talk about entirely different topics, and change one's perception of the world around, so it may end up with changing political decisions as well.
I am very speculative right now, but in this kind of topics that has huge amount of economical goals and need (like in medical drugs), research can be manipulated to support their work, and LLMs are extremely good at learning the given knowledge (I am currently doing research on blindness of LLMs on their training, so I think the risk bigger than you think). The world has countless cultures and ethical perspectives, and LLMs are trained by several groups, so we are moving towards trained models' decision distribution. You say studies only measure the checkable part, I think this indirect kind is even harder to measure, but it is real in society.
You also say persuasion needs trust and relationship, like in GPT-4o case. I think this trust is already there. I am in this bubble that I use AI daily since my job is AI, I am mostly aware what is happening, sadly, however, every non-technical people around me use AI even for free psychologist, or ground truth for their needs which means that people already adapting and escaping from reality, and letting it give decisions for their lives.
Persuasive AI is one of the classic AI risk scenarios. If AIs can talk people into believing or doing things they wouldn’t otherwise believe or do, that could seriously threaten elections and democratic institutions. In more dramatic scenarios, it’s a threat to human control over AI itself: before an AI can act in the world on its own, it could convince humans to act on its behalf.
We now have a growing pile of studies that tell us AIs are now extremely persuasive, more so than expert humans. But does persuasion in a study setting translate into real-world persuasion? It turns out the answer is no.
That gap matters beyond persuasion. The critical question today is whether AI progress on checkable tasks will generalize to fuzzy tasks. Because fuzzy tasks are by definition hard to evaluate, we have very little evidence to go off of. Persuasion is a rare fuzzy domain where we can actually check evaluations against real-world evidence. Whether AI progress extends to the fuzzy parts of real-world persuasion can tell us whether to expect it to extend to fuzzy tasks in other domains, like AI research.
So: where is all the AI persuasion?
On paper, frontier AIs are superhuman at persuasion
A long line of research has shown that AIs are frighteningly persuasive. Most recently, in June 2026, the UK AI Security Institute released a study in which frontier models were matched up against champion debaters and expert political canvassers to see who was better at changing people’s minds in text conversations. Across nearly 20,000 conversations on policy issues, AIs moved people’s views about 5 points more than the elite debaters and professional canvassers did. They even got people to give more real money to charity. The bottom line was that the best AIs now out-persuade the best humans we can find.
In another study, researchers had AIs talk with voters ahead of the 2024-25 American, Canadian and Polish national elections to see whether they could change minds in a high-stakes election. They could, with effects several times larger than those of typical political ads.
The earlier research on AI persuasion is summarized in a 2025 meta-analysis that pooled seven studies from 2023 to 2025. It found that AIs were, on average, about as persuasive as humans. But results varied wildly from study to study, and most of that variation came down to which model was used, whether the AI was having a conversation or writing a one-off message, and what topic it was arguing about – which is why the newest studies, with frontier models in live conversation, find AIs pulling ahead.
All of these studies together show a clear trajectory: a couple of years ago, AIs had caught up with ordinary human persuaders, and today, frontier models in live conversation beat the best humans we can find and move people’s views more than political ads do.
You get the point. I’ve dumped a bunch of studies on you, and they all point the same way: AIs have enormous persuasive power.
But there’s no evidence of AI persuasion in the wild
If we really have superhuman persuasion technology – available to every political campaign and intelligence agency in the world – then we should expect to see it working in the wild. Do we?
(I’m focusing on political and social media persuasion, which doesn’t cover every kind of AI persuasion. For example, an AI put in control of a platform could manipulate its recommendations. I’m interested in the specific kind of persuasion the studies measure, directly communicating information and opinions, because that’s what tells us whether the studies are misleading about real-world impact.)
The obvious place to start is the 2026 midterm elections. Campaigns spent at least $187 million (and probably more) on AI-generated ads. But they themselves acknowledged that this wasn’t very effective – a point underscored by the fact that Republicans did about 90% of this spending and got crushed. A few campaigns used AI chatbots for text-message canvassing – the medium where you might expect AIs to shine – but even this didn’t work. One campaign reported that only 1.7% of targeted voters responded to these chatbots at all, and of those, only 10-20% stayed engaged with the chatbot for an hour or more. That’s not nothing, but it is far from powerful.
Most compelling is the missing evidence of political persuasion at the aggregate level. If AI campaign messaging were changing minds, it should show up in the results as a late swing toward whoever was doing the messaging – a swing the polls would have partly missed. Since Republicans did most of the AI ad spending, you’d expect them to beat their polls. Instead, the polls underestimated Democrats by about four points in key races. Whatever moved voters in 2026, it doesn’t look like AI persuasion.
Another domain to study is foreign influence operations. Since 2024, OpenAI has been publishing reports on ChatGPT-powered influence operations run out of Russia, China and Iran. Except these operations are hilariously ineffective. In OpenAI’s first report, none of the five operations it documented scored above a 2 on the six-point Breakout Scale, meaning none of them gained traction with real audiences. Many operations faked their own engagement by having ChatGPT write replies to their own posts, since nobody else was replying. So even the most motivated and unscrupulous attempts at AI persuasion are ineffective.
Finally, anecdotal evidence: do you actually see any effective AI persuasion on the internet? I see a lot of bot accounts on Twitter, and I don’t see any signs of them having a persuasive impact. For example, the Midas Project found hundreds of sockpuppet accounts posting against export controls and arguing for Nvidia to be allowed to sell to China. But were these sockpuppets actually effective? Has anyone ever seen a random no-name account drop a Claudish one-liner like “it’s not about safety; the real question is whether we want America to control the AI tech stack” and changed their mind? I doubt it.[1]
So: whether you look at elections or social media, there is a shocking lack of AI persuasion in high-stakes settings.
What’s going on?
What the studies have in common is that they focus on narrow settings in which persuasion is uniquely easy. Subjects in an experiment engage with AI messaging by definition: they sign up and volunteer their attention to long, information-dense conversations. And the fact that they’re willing to participate at all suggests they don’t hold their beliefs very strongly. In studies, AIs get to write long, coherent paragraphs and are treated as a trusted source by default. This is easy-mode persuasion.
But real-life persuasion takes place under much more hostile conditions. A professional political canvasser has to stick their foot in your door before you slam it shut. On the internet, social media algorithms decide whether you see a persuader’s message and in what context, and even then you have to recognize the messenger and want to trust them. The Russian influence campaigns might have sounded great in theory, but they went up against people’s hardened information filters and failed to break through. That’s hard-mode persuasion, and it turns out AIs fail at it once you take away the protections of the lab.
Besides, the persuasion studies show a fair amount of “reward hacking”, so to speak. Whereas expert human persuaders rely on emotional appeals and building relationships, the studies show that AIs persuade by being information-dense, i.e. by dumping huge numbers of facts on people. But this is fake persuasion! It’s like when your annoying friend talks at 300 words per minute and cites 20 studies to prove you wrong, and it’s too overwhelming to process, so instead of arguing you say “uh huh” and move on with your day. This is exactly the kind of “persuasion” that gets people to change their minds in a lab study, then walk out the door and promptly forget all of it. So the studies are likely not even measuring real persuasion.
Are there other explanations for this gap between persuasion studies and the real world?
So AI persuasion is much less effective in real life than in evaluations, because evals provide ideal conditions. AI persuasion is a greenhouse plant: it thrives under controlled conditions, then wilts the moment it’s taken outside.
The eval-real world gap strikes again
This is a general problem with AI evaluations. We generally understand that evals only represent checkable tasks, and smart people pay lip service to this idea all the time. But we don’t appreciate how badly that distorts our picture of the domain an eval is supposed to measure. Every domain has some share of checkable and non-checkable tasks, and only the checkable ones show up in studies. In persuasion, it turns out that the checkable part (get people to report changing their mind after an interaction) is easy, while the non-checkable part (get your foot in the door so that people are willing to be persuaded) is all of the work. The case of AI persuasion pours cold water on the idea that we can simply extrapolate from domain evaluations to AI capability in that domain.
It’s not like we’ve never seen persuasive AIs before. GPT-4o was the most persuasive AI model ever. When OpenAI first tried to retire 4o last year, the backlash forced them to bring it back, and #keep4o is a hashtag that you can still find in Sam Altman’s replies. No other model has inspired anything close to that devotion. Of course, there’s a difference between sycophancy and persuasion. But the obsessive relationship that people established with 4o was fertile soil for 4o to persuade them to do anything.
So persuasion is one domain – like writing – in which AIs have arguably become worse. And as with writing, persuasion isn’t a single dial that turns up as models get smarter. Newer models pack more facts into every reply, which makes them look better in laboratory persuasion studies, but they don’t win devoted fans the way 4o did. So the changes that made AIs better at the part of persuasion we can measure… made them worse at the part we can’t measure. I doubt persuasion is unique here. Any capability that a bundle of checkable tasks and fuzzy tasks can get better and worse at the same time, but evaluations won’t show us when that happens.
To be fair, this excludes an important function that sockpuppets could have, which is to signal boost messaging from more prominent accounts/influencers. But that is not a sign of AI persuasion – even dumb bots can like and retweet with the best of us.