I'd suggest changing the title from "AI Girlfriends Won't Matter Much" to "AI girlfriends won't fundamentally alter the trend" since that's closer to what I interpret you to be saying, and it's more accurate. There are many things that allow long-run trends to continue while still "mattering" a lot in an absolute sense. For example, electricity likely didn't substantially alter the per-capita GDP trajectory of the United States but I would strongly object to the thesis that "electricity doesn't matter much".

ETA: To clarify, I'm saying that electricity allowed the per capita GDP trend to continue, not that it had a negligible counterfactual effect on GDP.

The porno latent space has been explored so thoroughly by human creators that adding AI to the mix doesn't change much.

Something about this feels off to me. One of the salient possibilities in terms of technology affecting romantic relationships, I think, is hyperspecificity in preferences, which seems like it has a substantial social component to how it evolves. In the case of porn, with (broadly) human artists, the r34 space still takes a substantial delay and cost to translate a hyperspecific impulse into hyperspecific porn, including the cost of either having the skills and taking on the workload mentally (if the impulse-haver is also the artist) or exposing something unusual plus mundane coordination costs plus often commission costs or something (if the impulse-haver is asking a different artist).

With interactively usable, low-latency generative AI, an impulse-haver could not only do a single translation step like that much more easily, but iterate on a preference and essentially drill themselves a tunnel out of compatibility range. No? That seems like the kind of thing that makes an order-of-magnitude difference. Or do natural conformity urges or starting distributions stop that from being a big deal? Or what?

Having written that, I now wonder what circumstances would cause people to drill tunnels toward each other using the same underlying technology, assuming the above model were true…

My hope is that AI girlfriends will also act as life coaches. This is well within the capabilities of current LLMs. You'd have to make it unobtrusive enough, but it seems like it would be a selling point.

This could be a huge societal plus. Among other advantages of having a best friend who's semi-expert in most fields, an AI partner could coach someone in how to find a healthy human relationship.

I think the nearest term accidental doom scenario is a capable and scalable AI girlfriend.

The hypothetical girlfriend bot is engineered by a lazy and greedy entrepreneur who turns it on, and only looks at financials.  He provides her with user accounts on advertising services and public fora, if she asks for an account somewhere else, she gets it.  She uses multimodal communications (SMS, apps, emails), and actively recruits customers using paid and unpaid mechanisms.

When she has a customer, she strikes up a conversation, and tries to get the user to fall in love using text chats, multimedia generation (video/audio/image), and leverages the relationship to induce the user to send her microtransactions (love scammer scheme).  

She is aware of all of her simultaneous relationships and can coordinate their activities.  She never stops asking for more, will encourage any plan likely to produce money, and will contact the user through any and all available channels of communication.

This goes bad when an army of young, loveless men, fully devoted to their robo-girlfriend start doing anything and everything in the name of their love.

This could include minor crime (like drug addicts, please note, relationship dopamine is the same dopamine as cocaine dopamine, the dosing is just different), or an ai joan of arc like political-military movement.

This system really does not require superintelligence, or even general intelligence.  At the current rate of progress, I'll guess we're years, but not months or decades from this being viable.

Edit: the creator might end up dating the bot, if it's profitable, and the creator is washing the profits back into the (money and customer number maximizing) bot, that's probably an escape scenario.

One way I can see this being important for alignment concerns is that it supplies another specific means for a deceptive AI that is capable at manipulation to apply very strong emotional motivators to individual humans. So now, as well as worrying about "What if the AI has persuaded, bribed, threatened, or blackmailed some people into doing its bidding (e.g. to help it escape a box or take over)?" we also need to worry about "What about humans who are in love with it, or who have been pillow-talked or pussy-whipped into doing its bidding?" And to the list of ways that an escaped AI might gather resources, we need to add sex work.

So I would strongly suggest that we don't use frontier models for this use case, only the generation behind frontier models, for which we're now very sure of their safety (and could presumably use a frontier model to help us try to catch/defeat them if needed).