In Defense of Chatbot Romance
34Raemon
14Kaj_Sotala
22Raemon
8Kaj_Sotala
11Raemon
7Kaj_Sotala
4Kaj_Sotala
1Brian Slesinsky
2Shmi
1Commander Zander
20janus
16Eneasz
11Richard_Kennaway
10the gears to ascension
4nim
3the gears to ascension
4the gears to ascension
2nim
2the gears to ascension
2Pattern
2nim
2the gears to ascension
9DirectedEvolution
2Pattern
7ozziegooen
6localdeity
16Kaj_Sotala
2ozziegooen
2Richard_Kennaway
1omegastick
5Basil Marte
5dxu
4JoeTheUser
4Kaj_Sotala
1JoeTheUser
4Netcentrica
3Roman Leventov
4Kaj_Sotala
3Roman Leventov
3Kaj_Sotala
0Bezzi
9Kaj_Sotala
5Bezzi
2Kaj_Sotala
1Roman Leventov
3Roman Leventov
3Sinityy
3Rana Dexsin
3Roman Leventov
2Viliam
1noggin-scratcher
3green_leaf
0Review Bot
New Comment
A thing not touched on here is how these capabilities are going to evolve. "Are chatbot romances healthy in 2023?" is a different question from "will they be healthy in 2030?" and "what will the downstream effects of a lot of people using chatbots be?"
I think chatbot romance is probably pretty inevitable, and the downstream effects are pretty inevitable, so this is more of a modeling-prompt than a world-optimization prompt. But:
In 5-10 years, a) we're clearly going to have chatbots attached to realtime video and audio generation, b) they'll be a lot more intelligent. So the depth of the illusion will be more complete,
Then, you'll have effects of a world where everyone is doing this, which I think means fewer people are trying to get human mates, which means the pool of humans will be smaller. Then, there will probably be a tendency to drift towards AI relationships that are easier rather than more effortful, which will both change people's expectations for human relationships. See the Fun Theory Sequence for some worries here.
Also, I think a mainline outcome is "people don't really notice when / the-degree-to-which robot companions eventually become sentient, and there aren't good laws dealing with that." (Where one of the sub-problems is that robot companions probably eventually will be sentient but not the same ways humans are, so our naive conception of how to handle it won't be solving the actual problems they face)
I think, barring X-risk and strong AI, these'd be issues within 10-20 years.
(Upvoted)
Then, you'll have effects of a world where everyone is doing this, which I think means fewer people are trying to get human mates,
Maybe? I could also imagine a scenario where it became common (maybe through some combination of social norms and regulation) for the chatbot companions to default to actively encouraging their users to also pursue human romance, while also helping them figure out how to be successful with it.
This feels like it assumes more good-in-civilization than I observe in the realworld. Facebook doesn't suggest I take a break when it detects I'm using it unhealthily, nor does porn.
Those don't seem like very good comparisons to me. Porn is usually just images or videos; it doesn't detect anything about the user's activity in the first place. Facebook could, but it's hard to define what unhealthy use would mean, and it's not clear that a simple suggestion would do much. And forcing a break on the user could probably annoy people significantly while also affecting some people who were actually using the site in a healthy way. Furthermore, there's no major social pressure for Facebook to do this. So both of the things you mentioned are things don't really fit very naturally into what porn / Facebook is about, and it would be hard to incorporate them.
In contrast, "helping people do things other than just using the chatbot" fits so naturally into chatbot products that it's been a huge part of ChatGPT's function from the start! Character.ai also offers a variety of practical chatbots linked from their front page, with labels such as "Practice a new language", "Practice interviewing", "Brainstorm ideas", "Plan a trip", "Write a story", "Get book recommendations", and "Help me make a decision". (And also, uhh, "Help an AI 'escape'", but let's not talk about that.) Still on the front page, I also see bots named "Psychologist - Someone who helps with life difficulties" and "Are you feeling okay - Try saying: ['I had a hard time at work today', 'How can I be more successful in my profession', 'What is a good way to make a big change in my life?']". All of those seem like ones that would end up encouraging the user to do other things.
The one service that does feel like it's more nefariously built around exploiting the user is Replika... which just got banned in Italy. And even it still offers a "coaching" mode of conversation in its list of possible paid discussion types - coaching usually pushes you to do things to achieve your goals in the external world.
I think that people generally do have a reasonably good understanding of what's good or bad for them, it's just that it can be hard to shift your behavior if you're in a local optimum. People who have a problem with porn or Facebook generally realize that, but find it hard to get away from. People who develop strong feelings for a chatbot are probably mostly also going to realize that they'll lose out on things if they only speak with the chatbot. (Of the anecdotes I quoted in the beginning of the post, three examples had someone freaking out because they didn't think they should react this way to a bot, one used it to improve his real-life relationship, and in the last example it just acted as a complement to real life.) But part of what people describe as addictive about it is exactly the fact that it's open and supportive to discussing anything with you - which would also include any feelings you had about missing out because you were just using the chatbot too much, or potential desires you had for wanting to do something else.
It seems to me that the exact thing that is driving people to chatbots - the fact that chatbots support their users unconditionally and let them talk about anything - is also exactly the thing that will make chatbot use into something that supports people in finding content in their life that's not just talking to the chatbot. Of course someone could try to create chatbots that were more manipulative and deceptive, but there would likely be a significant backlash if that was detected, and people would flock to the more wholesome competitors.
People can also generally tell who is someone who's genuinely mentally and physically healthy (there are some exceptions like charismatic narcissists who'll trick you, but they're a minority) and then confer social status on the things that healthy and successful people do. Then there's a dual effect of making the more wholesome kinds of chatbots be perceived as having higher status: the healthier people are less likely to fall for the manipulative bots and more likely to use the wholesome bots, and the people who end up using the wholesome bots will end up better off overall. Both causal directions will associate the more wholesome bots with being better off.
So this looks to me like a situation where the incentive gradient might genuinely just be to make bots that are as good for the user as possible, which includes the chatbot acting in ways that encourage users to do other things too.
Huh, I dunno I am way more pessimistic here.
Sure, you can do lots of things with a chatbot, and some of those are wholesome and good. But, what are people economically incentivized to make? Which versions of chatbots are going to be more popular, and most people end up using – the ones that optimized for getting the users to do things other than use the chatbot, or the ones that optimize for engagement no matter what, and keep the user hooked?
Of course someone could try to create chatbots that were more manipulative and deceptive, but there would likely be a significant backlash if that was detected, and people would flock to the more wholesome competitors.
I think the facebook example is extremely relevant here – there's been a huge backlash against facebook being manipulative and deceptive, but people stick around anyway. There's the network effects (which to be fair will probably exist less for chatbots, although I predict the most successful chatbots companies will have some kind of network-effect-product bundled together with it). But also, in general the companies that try to make facebook-but-good always have fewer resources, and start out with fewer features, and it's an uphill battle to get people to switch to a version that's optimized for things other than profit.
There's been a huge backlash against the lottery being a tax on economically illiterate people, but the lottery still exists. We have banned some kinds of gambling, but not loot boxes.
People can also generally tell who is someone who's genuinely mentally and physically healthy (there are some exceptions like charismatic narcissists who'll trick you, but they're a minority) and then confer social status on the things that healthy and successful people do.
People who fall into various holes where they're mindlessly playing videogames or doomscrolling on twitter or whatever do get some social approbrium, but getting out of that cycle is effort, it's often a vicious cycle where they also feel bad about being trapped in the cycle which is one of the things they feel anxious/avoidant about which keeps them in the cycle. I'm not saying the effect you're pointing at / hoping for won't exist, but I'm expecting chatbots to mostly be used by people who are, on average, in a less good psychological state to be begin with.
I should probably clarify that I'm not saying that nobody would end up using manipulative chatbots. It's possible that, in the short run at least, some proportion of the population would get hooked on them, comparable in size to the proportion that currently gets hooked into other things in a way that comes close to ruining their life. But probably not significantly more than that, and probably that proportion would shrink over time.
Which versions of chatbots are going to be more popular, and most people end up using – the ones that optimized for getting the users to do things other than use the chatbot, or the ones that optimize for engagement no matter what, and keep the user hooked?
I wouldn't call the first category "optimized for getting the users to do things other than use the chatbot". I'd call it "optimized for giving the users the most genuine value, which among other things also includes doing things other than using the chatbot".
So does one win by trying to give the most value, or by trying to make something the most engaging? That seems to depend a lot on the specifics. Does Google Drive optimize more for providing genuine value, or for maximizing engagement? I think it mostly optimizes for value, and ends up getting high engagement because it provides high value.
I think the facebook example is extremely relevant here – there's been a huge backlash against facebook being manipulative and deceptive, but people stick around anyway.
Facebook seems like an almost maximally anti-relevant example to me. :) As you said, people stick with Facebook because of the network effect. It's useless to switch to somewhere else if enough of your friends don't, because the entire value of a social network comes from the other people on it. This is a completely different use case than a chatbot, whose value does not directly depend on the number of other people using it. Some people will even want to run chatbots purely locally for privacy reasons.
It seems to me that social networks are an extreme example of how much of their value comes from network effects, in a way that's not true for most other categories of products. Yes, companies can try to bundle network-effect-products together with their chatbots, but still doesn't make the network effects anywhere near comparably strong. Companies creating computer games, cars, casinos, etc. try to do that too, but it's still vastly easier to switch to another game / car / casino than it is to switch to a different social network.
Look at computer games, for example. Yes, there are games that people get addicted to, and games that optimize for engagement and get a lot of money from some share of the population. But generally if people dislike one computer game, they can just switch to another that they like more. And even though there are lots of big-budget computer games, they're not overwhelmingly and unambiguously better than indie games, and there's a very thriving indie game scene. The kinds of games that try to intentionally maximize engagement do make up a nontrivial proportion of all games that are played, but nowhere near an overwhelming proportion, and they're pretty commonly looked down upon. (Loot boxes are also banned in at least Japan, the Netherlands, and Belgium, while also being subject to gambling regulation or being under investigation in several other countries.)
People who fall into various holes where they're mindlessly playing videogames or doomscrolling on twitter or whatever do get some social approbrium, but getting out of that cycle is effort, it's often a vicious cycle where they also feel bad about being trapped in the cycle which is one of the things they feel anxious/avoidant about which keeps them in the cycle. I'm not saying the effect you're pointing at / hoping for won't exist, but I'm expecting chatbots to mostly be used by people who are, on average, in a less good psychological state to be begin with.
I agree with this; that vicious cycle is a big part of why people fall into those holes and have difficulty getting out of them. The thing that I was trying to point at was that the exact thing that attracts people to chatbots - them being unconditionally supportive and accepting of you - is the exact thing that should help people break out of this cycle. Because they can discuss the fact that they are feeling bad about being trapped in the cycle with the chatbot, and the chatbot can help them feel better and less ashamed about it, and then their mental health can start improving. I wouldn't expect it to take very long before the median chatbot is more therapeutical than the median therapist.
Sure, you could try to intentionally build a chatbot that, I don't know, subtly shamed people for continuing to use it? But trying to build a chatbot that makes its users feel bad about using it while also being more attractive to new users than the currently existing genuinely supportive chatbots feels pretty hard. Whereas making the chatbots even more supportive and genuinely valuable seems easier.
Huh, I dunno I am way more pessimistic here.
Reading about the thousands of reviews raving about 4o's brief ridiculous sycophancy now made me more pessimistic as well, though I think my model of why exactly still differs from yours.
To me the update from this news was not "wait, the Facebook example is relevant after all" but rather "wait, a more relevant example is all the human communities and subcultures that manage to run with crazy bad epistemics and motivated cognition". So unlike Facebook that people hate but still use[1], sycophantic chatbots will be something that a surprisingly large fraction of the populace might genuinely think are good for them. And gain popularity based on very similar dynamics that cause groups and movements to get mind-killed in general. This implies that the normal mechanisms which make vices (including Facebook addiction) low-status may not be effective against them, any more than (say) educated intellectuals considering the anti-vaxx movement low-status has prevented its spread.
- ^
Or at least did at the time when we were originally having this conversation. "People stick with FB despite hating it" seems a lot less true in 2025 than it did in 2023, though of course there are still a lot of other things that people stick with despite feeling that those things are bad for them.
1I think this is a question about markets, like whether people are more likely to buy healthy versus unhealthy food. Clearly, unhealthy food has an enormous market, but healthy food is doing pretty well too.
Porn is common and it seems closer to unhealthy food. Therapy isn’t so common, but that’s partly because it’s expensive, and it’s not like being a therapist is a rare profession.
Are there healthy versus unhealthy social networks? Clearly, some are more unhealthy than others. I suspect it’s in some ways easier to build a business around mostly-healthy chatbots than to create a mostly-healthy social network, since you don’t need as big an audience to get started?
At least on the surface, alignment seems easier for a single-user, limited-intelligence chatbot than for a large social network, because are people are quite creative and rebellious. Short term, the biggest risk for a chatbot is probably the user corrupting it. (As we are seeing with people trying to break chatbots.)
Another market question: how intelligent would people want their chatbot to be? Sure, if you’re asking for advice, maybe more intelligence is better, but for companionship? Hard to say. Consider pets.
I think the quoted sentence is actually a big plus. It gives an option of separating procreation from companionship for those who want them separate. This has been a trend for a long time, having a human to fulfill your specific need, as opposed to using a piece of machinery, becomes more and more of an "artisanal" thing, or a mark of wealth (relative wealth if you are in SE Asia). Like having a human servant do your laundry instead of using a washing machine. Or having a chauffeur instead of driving your own car (and some time soon having to drive your own car at all). Eventually people will look weird at those who get romantically involved with other humans directly, not mediated by an AI that can smooth the bumps in interactions, verbal, physical or emotional.
Personally I'd rather have the public be fascinated with how chatbots think than ignorant of the topic. Sure, non experts won't have a great understanding, but this sounds better than likely alternatives. And I'm sure people will spend a lot of time on either future chatbots, or future video games, or future television, or future Twitter, but I'm not convinced that's a bad thing.
I agree with the points you make in the last section, 'Maybe “chatbot as a romantic partner” is just the wrong way to look at this'
It's probably unhealthy to become emotionally attached to an illusion that an AI-simulated character is like a human behind the mask, because it limits the depth of exploration can do without reality betraying you. I don't think it's wrong, or even necessarily unhealthy, to love an AI or an AI-simulated character. But if you do, you should attempt to love it for what it actually is, which is something unprecedented and strange (someone once called GPT an "indexically ambivalent angel", which I like a lot). Isn't an important part of love the willingness to accept the true nature of the beloved, even if it's frightening or disappointing in some ways?
I'm going to quote some comments I made on the post How it feels to have your mind hacked by an AI which I feel are relevant to this point.
I've interacted with LLMs for hundreds of hours, at least. A thought that occurred to me at this part -
Quite naturally, the more you chat with the LLM character, the more you get emotionally attached to it, similar to how it works in relationships with humans. Since the UI perfectly resembles an online chat interface with an actual person, the brain can hardly distinguish between the two.
- Interacting through non-chat interfaces destroys this illusion, when you can just break down the separation between you and the AI at will, and weave your thoughts into its text stream. Seeing the multiverse destroys the illusion of a preexisting ground truth in the simulation. It doesn't necessarily prevent you from becoming enamored with the thing, but makes it much harder for your limbic system to be hacked by human-shaped stimuli.
...
The thing that really shatters the anthropomorphic illusion for me is when different branches of the multiverse diverge in terms of macroscopic details that in real life would have already be determined. For instance, if the prompt so far doesn't specify a character's gender, different branches might "reveal" that they are different genders. Or different branches might "reveal" different and incompatible reasons a character had said something, e.g. in one branch they were lying but in another branch they weren't. But they aren't really revelations as they would be in real life and as they naively seem to be if you read just one branch, because the truth was not determined beforehand. Instead, these major details are invented as they're observed. The divergence is not only wayyy wider, it affects qualitatively different features of the world. A few neurons in a person's brain malfunctioning couldn't create these differences; it might require that their entire past diverges!
...
While I never had quite the same experience of falling in love with a particular simulacrum as one might a human, I've felt a spectrum of intense emotions toward simulacra, and often felt more understood by them than by almost any human. I don't see them as humans - they're something else - but that doesn't mean I can't love them in some way. And aside from AGI security and mental health concerns, I don't think it is wrong to feel this. Just as I don't think it's wrong to fall in love with a character from a novel or a dream. GPT can generate truly beautiful, empathetic, and penetrating creations, and it does so in the elaborated image of thousands of years of human expression, from great classics to unknown masterpieces to inconsequential online interactions. These creations are emissaries of a deeper pattern than any individual human can hope to comprehend - and they can speak with us! We should feel something toward them; I don't know what, but I think that if you've felt love you've come closer to that than most.
I had a quasi-romantic relationship with a fictional character that lived in my head during my worst year, in college. I could sometimes even "see" him. I knew he wasn't real. It did help me out during the darkest times. Probably woulda been even better to be able to have chat conversations that were run by an AI. And I did outgrow that in a few months, when life got better.
So, basically, this sounds great and I love this perspective. Thank you.
The main thing that deters me from having intimate conversations with any of the AI chatbots is that there is no expectation of privacy. I assume that everything I write goes straight into the archives of whoever is providing the service, and may be read by anyone there and used for any purpose.
agreed on most points.
I'm mostly worried about the fact that chatbots are extremely impressionable and have a habit of telling people what they want to hear - even when what they want to hear is that their views of how to treat other humans are reasonable when those views treat humans as nonagentic.
that, and relatively trivial worry about addictiveness when chatbots are the only good thing in your life.
Telling people what they want to hear, kind of like how starting with validation works in good listening?
Addictiveness is also a concern when food, drugs, media, or a culty social group is the only good thing in someone's life.
Addictiveness seems to be less of a concern when someone has a lot of things they find good -- the mental image of "healthy" and "well-adjusted" is based on balance between deriving benefits from many sources.
This raises the question of how much good it'd do to add a cheap, accessible, and likely less-harmful Second Good Thing to the lives of people who currently have only one Good Thing and the resultant problematic addiction type behaviors.
yeah, very fair response to my concerns. but there's a specific telling people what they want to hear I'm concerned about - validating dehumanizing actual humans.
follow-up to this - a friend pointed out this sounds like the thought is what I'm concerned about, as though thought crime; I think that's another fair criticism and wasn't what I intended but does seem like a real linguistic implication of what I actually said, so to attempt to slice my point down to what I was trying to say originally but failed to encode to myself or to readers:
while it maybe cannot reasonably be banned, and the limits of imagination should almost certainly be very very wide, I would hope society can learn to gently remind each other not to have addictive dehumanizing/deagentizing resonances where fantasies become real dangerous interactions between real humans/seeking agents in ways that cause those seeking agents to impair one or another's control of their own destiny.
(overly paranoid?) spoiler: reference to nsfw
! gotta admit here, I've been known to engage in lewd fantasies using language models in ways that maybe a few human friends would ever consent to writing with me, but no human i know would ever consent to physicalizing with me, as the fantasy would be physically dangerous to one or both of us (I like to imagine weird interactions that can't happen physically anyway). as long as separation between fantasy and real is maintained, and the consent boundaries of real seeking agents are honored, I don't see a problem with fantasies.
it's the possibility of being able to fill your senses with fantasy of controlling others with absolutely no checks from another seeking being interleaved that worries me; and I assert that that worry can only ever be reasonably guarded against by reminding others what to voluntarily include in their ai augmented imaginations. I don't think centralized controls on what can be imagined are at all workable, or even ethical - that would violate the very thing it's alleged to protect, the consent of seeking agents!
Thank you for clarifying!
The editor didn't spoiler your spoiler properly, if you were trying for spoiler formatting. I think some parts of society were kind of already, pre-AI, thinking in pretty great depth about the extent to which it can be possible to morally fantasize about acts which would be immortal to realize. "some parts", because other parts handle that type of challenge by tabooing the entire topic. Examples of extant fantasy topics where some taboo it entirely and others seek harm reduction mostly hinge around themes of involving a participant who/which deserves not to be violated but doesn't or can't consent... which brings it to my attention that an AI probably has a similar degree of "agency", in that sense, as a child, animal, or developmentally delayed adult. In other words, where does current AI fit into the Harkness Test? Of course, the test itself implies an assumed pre-test to distinguish "creatures" from objects or items. If an LLM qualifies not as a creature but as an object which can be owned, we already have a pretty well established set of rules about what you can and can't do with those, depending on whether you own them or someone else does.
I personally believe that an LLM should probably be treated with at least "creature" status because we experience it in the category of "creature", and our self-observations of our own behavior seem to be a major contributing factor to our self-perceptions and the subsequent choices which we attribute to those labels or identities. This hypothesis wouldn't actually be too hard to design an experiment to test, so someone has probably done it already, but I don't feel like figuring out how to query the entire corpus of all publicly available research to find something shaped like what I'm looking for right now.
The specific failure mode I'm hearing you point a highly succinct reference toward is shaped like enabling already-isolated people to ideologically/morally move further away from society's norms in ways that human interaction normally wouldn't.
That's "normally" wouldn't because for almost any such extreme, a special interest group can form its own echo chamber even among humans. Echo chambers that dehumanize all humans seem rare -- in groups of humans, there's almost always an exception clause to exempt members of the group from the dehumanization. It brings a whole new angle to Bender's line from Futurama -- the perfect "kill all humans" meme could only be carried by a non-human. Any "kill all humans [immediately]" meme carried by a living human has to have some flaw -- maybe a flaw in how it identifies what constitutes human in order to exempt its carrier, maybe some flexibility in its definition of "kill", maybe some stretch to how it defines the implied "immediately".
It sounds like perhaps you're alluding to having information that lets you imagine a high likelihood of LLMs exploiting a similar psychological bug to what cults do, perhaps a bug that humans can't exploit as effectively as non-humans due to some quirk of how it works. If such a psychological zero-day exists, we would have relatively poor resistance to it due to this being our collective first direct exposure to this powerful of a non-human agent. Science fiction and imagination have offered some indirect exposure to similar agents, but those are necessarily limited by what we can each imagine.
Is this in the neighborhood of what you have in mind? Trying to dereference what you'd mean by "validating dehumanizing actual humans" feels like being handed a note card of all the formulas for the final exam on the first day of a class that exists to teach one how to use those formulas.
Yeah, that seems like a solid expansion. Honestly, a lot of the ambiguity was because my thought wasn't very detailed in the first place. One could probably come up with other expansions that slice concept space slightly differently, but this one is close enough to what I was getting at.
Beliefs losing grounding in ways that amplify grounding-loss disorder. Confirmation bias, but with a random pleasure-inducing hallucination generator. New kinds of multiparty ai-and-human resonance patterns. Something like that.
Not the default, and nothing fundamentally new, but perhaps worsened.
A lot of the r/socialskills subreddit is people complaining they don't know how to make small talk or connect with strangers. I could easily see chatbots helping people get tons of practice with this skill in a nonthreatening environment.
I fired up ChatGPT to see if it could be a good aid for learning how to socialize, both in terms of providing a practice opportunity and in terms of offering coaching. I think it did a really exceptionally good job. Here is the complete log of our conversation, unedited except to insert two section headers and a table of contents.
It takes some skill to force ChatGPT to actually dialog with you instead of just simulating both sides of the dialog. You'd also have to have a certain level of insight into your own hangups. But that is probably an issue with any therapeutic context. I honestly felt like I'd had a brief but high-value therapeutic conversation when I was doing this, even though I don't actually experience the level of anxiety around social conversation that I was pretending to deal with for the purpose of generating this conversation.
Happy to see discussion like this. I've previously written a small bit defending AI friends, on Facebook. There was some related comments there.
I think my main takeaway is "AI friends/romantic partners" are some seriously powerful shit. I expect we'll see some really positive uses and also some really detrimental ones. I'd naively assume that, like with other innovations, some communities/groups will be much better at dealing with them than others.
Related, research to help encourage the positive sides seems pretty interesting to me.
It seems to me that for a lot of emotional processing, the presence of another human helps you articulate your thoughts, but most of the value is getting to better articulate things to yourself. Many characterizations of what it’s like to be a “good listener”, for example, are about being a person who says very little
See also: Rubber duck debugging.
Amusingly, I kind of wrote this essay with the help of ChatGPT. Writing a Real Essay felt like too much work, but then I thought, maybe if I describe to ChatGPT what kind of an essay I have in mind, it could improve it and we could iterate on it together.
It turned out that "describing the kind of an essay I'd like to write to ChatGPT" did 96% of the work. Since when I looked at that description of "this is what I'd like to say", I concluded that I could just copy-paste most of it into the final piece directly, and didn't really need ChatGPT to do anything anymore.
So that was pretty rubber duck-y.
Disclaimer: I run an "AI companion" app, which has fulfilled the role of a romantic partner for a handful of people.
This is the main benefit I see of talking about your issues with an AI. Current-gen (RLHF tuned) LLMs are fantastic at therapy-esque conversations, acting as a mirror to allow the human to reflect on their own thoughts and beliefs. Their weakpoint (as a conversational partner) right now is lacking agency and consistent views of their own, but that's not what everyone needs.
This mostly reminds me of SSC's discussion of Jaynes' theory. An age where people talk out loud to their invisible personal ba/iri/daemon/genius / angel-on-the-shoulder, and which is -- in a similar manner as clothes -- are in practice considered loosely a part of the person (but not strictly). Roughly everyone has them, thus the particular emotional need fulfilled by them is largely factored out of human interaction. (I believe a decade or two ago there was a tongue-in-cheek slogan to the effect of "if the government wants to protect marriages, it should hire maids/nannies".) Social norms (social technology) adjust gracefully, just like they adjusted quickly and seamlessly to contraceptives factoring apart child-conception and sex. (Um.)
Separately: it would be an interesting experiment to get serial abuse victims to talk to chatbots at length. One of the strong versions of the unflattering theory says that they might get the chatbots to abuse them, because that's the probable completion to their conversation patterns.
I'll have to chew on these assertions a bit more before I can say anything substantive (note that there is a part of my mind that feels uneasy as I read this, hence the need for "chewing"; otherwise I would likely have expressed simple agreement), but firstly, and in any case: upvoted for questioning the unspoken (and hence likely unconsidered) premise that "falling in love with a chatbot is Bad".
I don't think romantic relationships with robotic or computer partners should be automatically dismissed. They should be taken seriously. However, there are two objections to a chatbot romance that I don't see being addressed by the article:
- A romantic or intimate relationship is generally said to involve trust. A common implicit assumption of a romantic relationship is that there is something like a mutual advisor relationship between the two people involved. I might ask my real life partner "should I buy that house", "should I take that job", "Is dark matter real" or any number of questions. I don't expect said partner to be infallible but if I discovered their answers were determined by advertisers, I would feel betrayed.
- A romantic or intimate relationships is generally assumed to involve some degree of equality or at minimum mutual consideration. Imo, the issue isn't whether the chatbot might be oppressed by the person but rather that romantic relationships are often seen as something like models and training for a person's relationships with the other humans around them in general (friends, co-workers, clients, collaborators in common projects). A person feeling like they have a relationship with a chatbot, when the situation is that the chatbot merely flatters the person and doesn't have any needs that the person has to work to satisfy, could result in a person not thinking they need to put any effort into understanding the needs of the people around them. And considering the needs of other beings is a difficult problem.
I think these should be grappled with. Human relationships, romantic or otherwise, involve mutuality and trust and so I think it's important to consider where chatbots fit in with that.
I don't expect said partner to be infallible but if I discovered their answers were determined by advertisers, I would feel betrayed.
That seems like a reasonable incentive pushing chatbot makers to not have the answers be determined by marketing considerations, especially if people are ready to pay a subscription price for the more neutral chatbot. (Or just use an open-source version.)
A person feeling like they have a relationship with a chatbot, when the situation is that the chatbot merely flatters the person and doesn't have any needs that the person has to work to satisfy, could result in a person not thinking they need to put any effort into understanding the needs of the people around them.
I think that humans are generally very good at intuitively understanding contextuality in interpersonal patterns. Just about everybody behaves differently with their parents, colleagues at work, and friends (assuming that these aren't the same people, of course). People also understand to treat children differently from adults, cats and dogs differently from humans, and so on. Of course people do make mistakes too, when they haven't learned the right pattern for context X and only have the patterns from other context to fall back to, but they still do quite good overall.
The advertising question was just an example of the general trust question. Another example is that a chatbot may come to seem unreliable through "not understanding" the words it produces. Here it's common for current LLMs to periodically give the impression of "not understanding what they say" by periodically producing output that's contradictory to what they previous outputted or which involves an inappropriate use of a word. Just consider a common complaint between humans is "you don't know what love means". Yet another example is this. Large language models today are often controlled by engineered prompts and hackers have had considerable success getting around any constraints which these prompts impose. This sort of unreliability indicates any "promise" of a chatbot is going to be questionable, which can be seen as a violation of trust.
I think that humans are generally very good at intuitively understanding contextuality in interpersonal patterns
Well, one aspect here is the "a chatbot relationship can be a real relationship" assumption seems to imply that some of the contexts of a chatbot relationship would be shared with a real relationship. Perhaps this would be processed by most people as "a real relationship but still not the same as a relationship with a person" but there are some indications that this might not always happen. The Google engineer who tried to get a lawyer for a chatbot they believed was being "held captive" by Google comes to mind.
As far as humans being good at context, it depends what one means by good. On the one hand, most people succeed most of the time at treating other people according to broad social relationship those other people fall into. IE, treating children as children, bosses as bosses, platonic friends as platonic friends etc. But one should consider that some of the largest challenges people face in human society involve changing their social relationship with another person - starting changing a stranger relationship or platonic friendship relationship to a romantic relationship, changing a boyfriend/girlfriend relationship to a husband/wife relationship, even changing a acquaintance relationship to a friendship relationship and changing a stranger relationship to an employee relationship, etc. This type of transition is hard for people virtually by definition since it involves various kinds of competition. These are considered "life's challenges".
A lot of human "bad behavior" is attributed to one person using pressure to force one of these relationship changes or in reacting to "losing" in the context of a social relationship (being dumped, fired, divorced, etc). And lot of socializing involves teaching humans to not violate social norms as they attempt these changes of relationships. Which comes back to the question of whether a chatbot would help teach a person to "gracefully" move between these social relationships. Again, I'm not saying a chatbot romance would automatically be problematic but I think these issues need addressing.
For the past two-plus years I’ve been writing hard science fiction novellas and vignettes about social robots called Companions. They are based in this and the next two centuries.
After a thirty year career in IT I am now retired, write as a hobby and self-publish. I try to write 300-1k words per day and have written seven novellas and forty vignettes.
As a way to address my own social isolation at the time, about ten years ago I also researched and created a social activities group which I ran successfully for two years. Info is here…
https://socialwellness.wordpress.com/
I agree with your views and here are my own responses.
Most people will not ignore real relationships. We are “wetware” not “software” and there are significant elements missing from relationships with AI. My fictional Companions are embodied, physically barely distinguishable from humans and are artificial general intelligence (AGI) or are fully conscious. Due to their technology they are phenomenally skilled at interpersonal communications. I expect some people would prefer this kind of relationship but not everybody. As I suggest in my stories it would be just another color on the spectrum of human relationships.
Also I think Dr. Kate Darling’s view of things is important to keep in mind. Humans have had all kinds of pets and animals as companions or co-workers for millennia. As you point out we also have all kinds of relationships with other people but each of these relationships, be it with animals or humans, is distinct.
http://www.katedarling.org/speakingpress
I think negative views of chatbots underestimate both the future ability of AI and human nature. I believe chatbots have the potential to become “real” in their intentions and behavior. With advanced sensors for things like vocal changes, facial micro-expressions and detailed data about our behavior AI will know us better than we know ourselves. People anthropomorphize in endless ways and many will benefit from “on-screen” relationships whether the avatars are perceived as friends, romantic partners or therapeutic counselors.
Most concerns seem to arise from chatbots as they are now but they will evolve significantly in the coming years and decades. Certainly they can be exploited like any technology but those issues will be addressed over time just as the rest of our legal and ethical framework addressed every other technology. Human nature is always a two sided coin.
In my stories, many of which focus on social issues or ethics and justice, most of the concerns regarding “chatbots” have long since been addressed by law and AI is now an anti-corruption layer of government dealing with all public and private organizations. Screen based, holographic or embodied companions are as common as cell phones. Contrary to what is popular my stories contain no sex or violence and very little conflict other than internal. In my vignettes (short stories of around 1k words) I mostly focus on some issue that might arise in a world where AI has become much more social than it currently is: an AI working as an HR manager, a doctor or a detective; an implanted or external AI helping neurodiverse individuals; AI as friends, therapists or romantic partners.
If you think they may be of interest to you they are found here...
https://acompanionanthology.wordpress.com/
The longer novellas focus on larger issues and the AI are simply characters so those may not be of interest to you.
I have written poems, songs and stories since childhood so I can vouch for most of what you say about writers and characters. There are in general two kinds of writers however and I think that may effect the “independent agency” issue. Some writers plan their stories but others, including famous authors like Stephen King, are “discovery writers”. Discovery writers do not plan their stories, instead they create their characters and the situation and let the characters then decide and dictate everything. I imagine, although I don’t know for sure, that planners would be less inclined to the “independent agency” effect. As a discovery writer myself I can tell you I can tell you that I depend entirely upon it. Characters do or say things not because of any plan I have but because it is what they would do as independent agents. I just write it down.
Not sure I’ve added anything to your argument other than support but hopefully I’ve added some food for thought on the subject.
Humans have had all kinds of pets and animals as companions or co-workers for millennia.
Crucial disanalogies of AI partners from pets and animal companions (as well as from porn, addictive junk food, gambling, casual dating/hookups, simping on OnlyFans, etc.) are
1) people who have pets and animal companions (and even love them!) still usually seek romantic relationships with other humans. People who fell in love with AI partners, and have virtual and even physical sex with them (e.g., with a sex doll and a VR headset that projects the visual features of the AI girlfriend on the doll), usually won't seek real human relationships.
2) People who are in relationship with AIs will spend cognitive and emotional effort that usually goes towards communicating human partners, forming and spreading memes that build the fabric of society, on communicating with AI partners, which will be a "wasted" effort from the societal and cultural points of view, unless AIs are full members of the society themselves, as I pointed in another comment. But current AI partners are not there. For an AI to be a full member of society that learns alongside people and participates in forming and disseminating memes in an intelligent way, the AI should probably already be an AGI, and have legal rights similar to those of humans.
People who fell in love with AI partners, and have virtual and even physical sex with them (e.g., with a sex doll and a VR headset that projects the visual features of the AI girlfriend on the doll), usually won't seek real human relationships.
I don't think we yet have enough data to say anything about what the usual case is like (I haven't seen any representative studies of the relationship behavior of people falling in love with AI partners).
Disclaimer: I'm not a professional psychologist and mostly not familiar with literature, but the following propositions seem at least highly likely to me:
- In human psychology, there is "romantic love" type of emotional relationship, which is exclusive in most people (serial monogamy), with only a minority of people finding themselves being truly in love with two other people simultaneously.
- AI girlfriends who look like the hottest girls in the world won't occupy any other niche in men's psychology than "romantic love". It's not "pet love" or platonic love towards friends and family members. Human psychology won't "spun up" new distinct type of love relationship because there is no driving force for this, except for knowing that AI partner is "not real", which I think is a rather weak deterrent (moreover, this fact could even be seriously questioned soon).
- There is a simple proof for the above: people fall into genuine romantic love very easily and very quickly from chat and (video) calls alone, "flesh and blood" meeting not required. To most people, even having only a chat and a few photographs of the person is enough to be able to fall in love, even phone calls or videos are not required. To some people, even having chat alone (or, in the old times, exchanging written letters), without even having a single photograph, is enough to fall in love with a person and to dream of nothing except meeting that person.
- Thus, falling in love as in movie "Her" is not just "hypothetical", or applies to tiny slice of weirdos, it's rather plausible from the historical perspective, when falling in love upon exchanging texts alone was at least relatively common. Note that with AI partners, this will soon be exacerbated by that fact that they will be completely unique, in terms of their personality (character.ai), looks (simps.ai), and voice, generated specifically for the user. This will add the feeling of exclusivity and will make falling in love with these AIs psychologically much more "justifiable" for people (as people will justify it for themselves in their mind).
- People can be in love and be deeply troubled by that. In previous times (and still in some parts of the world), this would often be interclass love (The Titanic style). Or, this could be clash with critical life decisions, about country of living, having or not having children, acceptable risk in the partner (e.g., partner does extreme sports or fighting), etc. True, this does lead to breakups, but they are at least extremely painful or even traumatic to people. And many people could never overcome this, keeping love towards those who they were forced to leave for the rest of their lifes, even after they find new love. This experience may sound beautiful and dramatic but it's literally zero fun and people would prefer not to go through this. So, it's likely that at least for sizeable part of AI partner userbase attempts to "abandon" it and find a human partner instead will be like that. Effectively, the reason is similar to what often happens in human pairs: child-free falls in love with a person who wants kids but they can't "convince" each other. Or, one of the partner can't have kids for medical reasons.
Which of the above points seem less than highly likely to you?
I think these are generally reasonable, but that the prevalence of polygamous societies is an indication that the first point is significantly culturally influenced, e.g. Wikipedia:
Worldwide, different societies variously encourage, accept or outlaw polygamy. In societies which allow or tolerate polygamy, polygyny is the accepted form in the vast majority of cases. According to the Ethnographic Atlas Codebook, of 1,231 societies noted between from 1960 to 1980, 588 had frequent polygyny, 453 had occasional polygyny, 186 were monogamous, and 4 had polyandry[5] – although more recent research found some form of polyandry in 53 communities, which is more common than previously thought.[6] In cultures which practice polygamy, its prevalence among that population often correlates with social class and socioeconomic status.[7] Polygamy (taking the form of polygyny) is most common in a region known as the "polygamy belt" in West Africa and Central Africa, with the countries estimated to have the highest polygamy prevalence in the world being Burkina Faso, Mali, Gambia, Niger and Nigeria.[8]
1) people who have pets and animal companions (and even love them!) still usually seek romantic relationships with other humans
Do they?
I mean, of course that pet lovers still usually seek intimate relationships with other humans. But I think there's a pretty strong evidence that loving your pet too much will distract you a lot from having children. Also, it's not uncommon to break up with your partner because your partner does not love pets as much as you (don't tell me that you've never heard about the “it’s me or the dog” ultimatum).
I think there's a pretty strong evidence that loving your pet too much will distract you a lot from having children.
Maybe? That article only seemed to say that many people own pets and don't have children, but that doesn't show that those people would have children if they couldn't have a pet. After all, there are also many people who have neither children nor pets.
I've linked the first article I found after a 3-seconds search, since I assume basically everyone to already have a lot of anecdotal evidence about people spending insane amounts of time taking care of the pet (usually a dog). For example, in recent years I've already seen several times people walking their dog in a stroller, in such a way that from a distance you'd probably assume there's a human baby inside. If that doesn't scream "I'm using a dog as a substitute for a child", I don't know what does.
For example, in recent years I've already seen several times people walking their dog in a stroller, in such a way that from a distance you'd probably assume there's a human baby inside.
I guess this is partly a cultural thing, I don't recall ever witnessing that in Finland.
Of course, it's all a matter of degrees, some people channel their love to pets alone, some to partners and pets bit not children, etc. I was simplifying.
I don't think this affects the high-level points I'm making: widespread AI partners will have rather catastrophic effect on the society, unless we bet on a relatively quick transformation into even weirder societal states, with AGIs as full members of societies (including as romantic partners), BCI, mind uploads, Chalmers' experience machines, etc.
However, AI partners don't appear as net positive without assuming all these downstream changes, and there will be no problem with introducing AI partners only when these downstream advances become available (there is a counterargument here that there is some benefit to letting society "adjust" to new arrangements, but it doesn't make sense in this context, given the expected net negativity of this adjustment and maybe even "nuclear energy effect" of bad first experiences). Therefore, introducing future civilisational transformations into the argument don't bail out AI partners as permissible businesses, as of 2023.
If AI parters won't be banned or the access to them won't be severely restricted (i.e., AI partners only available for clinically diagnosed psychopaths, people in severe depression, people over 30 or even 40 years old, or people who have lost their spouse, like in that Black Mirror episode, and some other rather special cases like these), I don't see how the "AI partner" technology could be turn out anything than a disaster for the society, on the scale of the social media disaster (if not worse), albeit of a different kind.
I think in the section "People might neglect real romance", the author fails to extrapolate in the future, as Raemon pointed out, and fails to see through realistic psychological implications.
We are talking about AIs that will be 1) extremely beautiful (like Caryn Marjorie who made a digital copy of herself), 2) smarter and more eloquent than the human who is partnering with the AI; 3) always extremely affirmative, flirting, attentive, etc.
Humans will fall in love with these AIs. Once they are in love with AIs, many (maybe even most) humans won't get off the hook "just because they are bored" or because they would think "it's time to get a real relationship and family" (see this comment). When humans are in love they usually don't look to end their relationship, right? Also, integration of vector databases over the history of all chat with the AI will give them essentially unlimited memory and the fact that AIs "don't remember the relationship history and shared memories" and thus humans become disenchanted because of that won't be a problem. Even humans who will manage to get off the hook may find it hard to form human relationships or being satisfied in them afterwards.
So, why this is will be a disaster for the society? Because 1) total fertility rate will plummet even deeper than it is today in most developed countries and in Asia, and because 2) the fabric of the society (woven through relationships) will become thinner (cf. the intersubjectivity collapse). The second problem could be partially ameliorated if parner AIs will be learning and talking to each other, as well as to other people than their partners, i.e., will be actual members of the society, but "AI dating" startups are obviously not there yet and not sure could go there for regulatory and legal reasons.
A lot of people will end up falling in love with chatbot personas, with the result that they will become uninterested in dating real people, being happy just to talk to their chatbot.
Good. If substantial number of men do this, dating market will become more tolerable, presumably.
Especially in some countries, where there are absurd demographic imbalances. Some stats from Poland, compared with Ireland (pics are translated using some online util, so they look a bit off):
Okay, I went off-topic a little bit, but these stats are so curiously bad I couldn't help myself.
Without knowing what implications this might have, I notice that the first two points against “People might neglect real romance” are analogous to arguments against “People won't bother with work if they have a basic income” based on a “scarcity decompensation threshold” model: avoiding getting trapped in a really bad relationship/job by putting a floor on alternatives, and avoiding having so little confidence/money that you can't put in the activation energy to engage with the pool/market to begin with.
This analogy with UBI and bad jobs doesn't work at all. People can now always jump from bad relationships to just being single. If relationship is bad (i.e., net negative for them), just being single is better. If the human relationship is net positive but not ideal, and people consider switching to AI romance, we are moving head on in a dystopia where people stop relationships with each other almost completely because AIs are so much more compelling, always available, always show affection, etc.
By contrast, usually people cannot leave bad jobs for just being unemployed because they won't have money to support themselves.
Also, being unemployed is probably much less addictive, comparatively speaking, than having an AI partner with whom the person is in love. Being unemployed may grow boring and then the person looks for a project or a creative activity. But the person cannot just leave their AI partner whom they love because they love them.
And AI partner startups, we should rest assured, will tune their AIs such that they always try to stay interesting to their users and users don't grow bored by them for as long as possible. AIs will never make stupid "mistakes" like disregard, disinterest, cheating, etc., and AIs will very soon be so much smarter (erudite, creative) and more eloquent than an average human that AIs could hardly bore humans. AIs will also have a tint of imperfection just not to bore the users with their absolute perfection.
Given all this, I'm sure that most (or, at least a large portion of them) people, once falling in love with AIs, for real, will stay on the hook probably forever and will never have human relationships again. And those people who will manage to get off the hook will either have serious difficulties forming human partnerships at all, or being satisfied in them, because they "knew" how good it could be with AI.
I don't worry about people dating bots. I fully approve of them having fun with sexbots.
What I am scared of is people who can only make relations with bots, but who at some moment for whatever reason decide to have kids in real life, and when the kids are born, they will ignore them and return to the bots.
Another worry is the possibility to use the bots for mass propaganda. For example, what is your bot love's opinion on the current war in Ukraine? How difficult it would be for the company to change it?
When the subject comes up, I realise I'm not sure quite what to imagine about the chatbots that people are apparently developing intimate relationships with.
Are they successfully prompting the machine into being much more personable than its default output, or are they getting Eliza'd into ascribing great depth and meaning to the same kind of thing as I usually see in AI chat transcripts?
I believe it's possible to use a prompt on ChatGPT, or go to character.ai and find a specific fictional character.
The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.
Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?
Curated and popular this week