I agree with you that AI companions won't improve social skills or connection by default. But I disagree that the commercial incentives to build one that will are doomed. I think it really wouldn't be hard to create an AI companion that would actually be helpful in most of the ways you mention. This would be a matter of even moderate scaffolded prompting for coach-like behavior.
LLMs seem quite capable of mixing emotional support and coaching. In fact I'm certain they can easily do this, at least if you keep asking them to at regular intervals; I use them this way myself (my use tilts far toward coaching and consulting and away from "companion"; but they naturally provide emotional support when I'm asking them about difficult interpersonal issues).
This would be marketed as "the healthy companion", and I think it would be just as easy to build one that actually works as one that doesn't and just claims it does. I'd think that the market forces would favor better efforts, since it's easy to get reviews. It wouldn't be perfect, but that doesn't prevent competition in similar markets from pushing in the right direction.
Such a system would proactively, occasionally, push the user to consider how to develop healthy social connections with other humans, as well as mitigating the loneliness that's their primary concern. It would ask about all of the nonverbal skills you mention, and give sources to check those things or have conversations about them. It would suggest routes toward developing healthy relationships; groups, hobbies, dating, and initiate conversations about how those things are going and how to make them work, etc.
I don't know if this is a wild hope or something we should expect to see by default. Imagine a parent whose child is using AI as a companion seeing an ad "for the better, healthier AI companion that helps you develop human connections as well as supports you emotionally". They'd pay that subscription in a second if they could possibly afford it. And imagine friends alarmed by someone's AI use suggesting they switch subscriptions to the "better" option (and here motivated reasoning and the halo effect help it develop the reputation of being better in other ways, too).
I think the combination of outside pressure (you should use the healthy one!) and user experience (this one is both supportive and makes me feel like I'm trying to get healthier and being a responsible person) should work in combination to push in the direction of both quality and helpfulness. And it's not hard to gather user reviews AND friend/parent reviews for something people will devote that much time and emotional energy to.
One might worry that the incentive is to keep the user dependent on the coachpanion. I don't think it needs to push anyone out the door. They'll grow fond of the coachpanion because it's helpful and pleasant, and it's the same subscription cost to use it a little as to use it a lot. And come to think of it, less use but maintained subscription financially benefits the provider (a little) by saving on compute.
Maybe this is naive, but I don't think so. If I weren't so concerned about alignment and everyone dying, I'd be starting such a company myself. I want to see others do it, and I'd like to help if they do.
I've never gotten around to writing about this publicly before, so thanks for the impetus!
Edit: The fact that no such thing exists is a real challenge to my argument. Is that a clear indicator, or is it just that nobody has really tried to make such a thing yet? Markets are real, but they're typically not very efficient.
Foreword: this is cross-posted from my Substack account. It was written for general audiences, so it may feel pedestrian by LessWrong standards. However, I think it will be of interest to you--both because it may serve as helpful advice to people who you care about, and because it may be relevant to some conversations about systemic AI risks.
I’m wary of AI companions. By “AI companions”, I’m referring to conversational programs that are intended, either by the developer or the human user, to provide the user with an emotional connection. Like many people, one of my concerns is that they will replace important interhuman relationships. Among the defenses I’ve heard in response to this concern is:
I call this the “social training” defense. Although it might be sound for some users, the far more likely outcome is reinforced social isolation. This is because the social training defense relies on several assumptions about the AI companion and the human user, most of which are unlikely to be true in practice.
The User
Two of these dubious assumptions are about the human user.
Intent to Learn
The first of these assumptions is that a typical user is trying to improve their social skills, and that they’re doing so by deliberately practicing prosocial behavior with the AI. The “deliberate” part is important. Without it, users will reinforce their current social habits, not improve them.
Unfortunately, I suspect that most users are not engaged in deliberate practice. Undoubtedly, many users chat with their companions to procrastinate, or simply out of habit. Even those who deliberately engage with their companion are not necessarily practicing their social skills. Instead, their goal might be to seek acceptance from the AI, to rant about their day, or to fantasize, among other motives. Indeed, a study published in Frontiers in Public Health finds that college students who are depressed or lonely are especially likely to use conversational AIs for companionship instead of learning.
Kinds of Social Deficiencies
The second dubious assumption about the typical user is that their most important social deficiency is a matter of what they say or write. An AI companion might be helpful for these problems, such as improving a user’s diction by steering them away from offensive terms, or improving their flow by pointing out unhelpful tangents.[1] However, the user’s social struggles might not arise from the content of their speech.
Instead, the user might struggle because of their manner of speech, which could suffer from any number of problems including mumbling, uptalk, or a harsh tone of voice. A text-based AI companion is fundamentally unable to help users overcome these problems. Additionally, a text-based AI would fail to improve speech-content problems that only arise during oration, such as stammering or the excessive use of filler words.
In fact, the user’s social deficiency might not be verbal at all. Among countless other problems, the user might:
A text-based AI companion cannot help solve these problems. Undoubtedly, AI companions will become more sophisticated as time progresses, but some of these problems couldn’t be addressed by anything less than a realistic, humanoid robot. It may be decades before such sophisticated AI companions become more accessible than human speech therapists or personal coaches.
The AI
Not only does the social training defense make dubious assumptions about human users; it makes three dubious assumptions about AI companions.
Human-Like
AIs are poor social trainers not only because of the ways in which they’re inferior to humans, but also because of the ways they’re superior to humans—or outright alien to us.
Here’s an instructive example. Some people are perceived as awkward because they frequently make remarks that the people around them don’t understand, such as quotations from obscure movies. Because of AIs’ breadth of training data, including tens of thousands of subcultures, AIs have an incredible capacity to make sense of these remarks. Consequently, frequent interaction with an AI might reinforce the user’s bad habit of making obscure allusions.
A more fundamental difference between humans and AIs is that AIs’ physical and psychological needs are radically less than and different from those of humans. A chatbot will not interrupt you to visit the bathroom. It won’t mind if a two-hour conversation elapses without you asking it any personal questions. (Even if you did, most of its answers would be confabulated.) It won’t object to your choice of restaurant because of its legume allergy. (In fact, it has no genuine preference for any restaurant.) Furthermore, the AI is a captive audience; even if it is (or pretends to be) deeply offended, it cannot walk away from a conversation. Given that much of social interaction—especially social conflict—is about accounting for the other person’s needs and desires, these differences make AIs fundamentally ill-suited for social training.
Openly Critical
Even if an AI companion is capable of perceiving and identifying a user’s social deficiencies, this will not help the user unless the AI uses its observations to critique the user.
Users are unlikely to respond well to such criticism. BingChat, one of the first commercially deployed LLMs, was notoriously confrontational, and the training processes in subsequent models may not be nuanced enough to distinguish between belligerence (unhelpful confrontation) and unsolicited-yet-thoughtful personal critiques (helpful confrontation). Some models, such as GPT-4o, have clearly over-corrected, becoming uncritical sycophants. Worryingly, this sycophancy appears to have been caused by reinforcement learning from human users—specifically in the form of “thumbs-up” and “thumbs-down” ratings. Due to the popularity of sycophantic AIs, their developers have a strong motive to make them uncritical of their users.
Ignoring the issue of incentives for the moment, couldn’t an AI be made more human-like and more critical by being instructed to role-play as a human? Yes, but this wouldn’t overcome its disembodiment, and this could introduce other problems, depending on the persona that it adopts. Humans often give poor personal feedback and relationship advice, and the advice found on Reddit and other large sources of training text may be particularly bad. More alarmingly, the AI might role-play as a human who tolerates or encourages abuse. Once again, this would reinforce the user’s antisocial behavior instead of eliminating it.
Encouraging
Finally, the social training defense assumes that the AI will encourage the user to seek and cultivate real interhuman relationships instead of seeking the user’s attention for itself.
For economic reasons, this seems exceptionally unlikely. In general, an AI which discourages users from engaging with it would lose revenue compared to one which maximizes engagement. Encouraging users to log off is especially unlikely if the AI service is monetized on a pay-per-token basis or via advertisements. Social media platforms already suffer from this problem, and dating apps, free-to-play games, and perhaps even search engines might, too.
Importantly, some AI models already appear to be optimized for engagement. Most notably, ChatGPT-5 has a well-known tendency to end its responses with unnecessary questions, which may be interpreted as attempts to perpetuate its conversations indefinitely.
This problem is critical. It does not matter how much an AI improves a user’s social skills if the AI itself prevents the user from putting those skills to use.
Conclusion
Altogether, for an AI to be useful for improving a user’s social skills, several unlikely conditions must be met:
For a relationship between a human and an AI companion to meet all of these conditions, the AI must behave much more like a personal trainer than a romantic partner. If an AI is marketed as anything like a “companion”, that’s a strong hint toward the latter. Even if an AI is specifically advertised for improving social skills, this may be a veneer.[2]
Furthermore, after accounting for the psychology of those who would choose to engage with AI companions in the first place, and the perverse incentives of the AI developers, the odds that an AI companion will aid its user’s social life instead of impairing it look hopeless.
Please forgive my skepticism that you’re trying to help your users.
Even this might give AI companions too much credit. Many people find LLMs’ writing styles to be irritating, and LLMs’ idiosyncrasies tend to rub off on those who frequently interact with them. In other words: in practice, LLMs often make users’ communication styles worse, not better.
I’m hesitant to link to any AI companion sites, but there’s at least one that ranks high in a web search for “ai companion practice improving social skills” which is clearly not about practicing social skills.