New Comment
Here, Meta just irresponsibly rolled out shitty hallucinating bots that encourage people to meet them “in person.”
I am sure that was unintentional; Meta doesn't want people to leave their screens and start meeting in real life.
1Thanks for the detailed analysis, Zvi.
But it’s sad, isn't? Despite developing technologies with such incredible scale, we are using them to amplify our lack of incentives to build a better future, rather than actually transforming it.
The conditions are: Lol, we’re Meta. Or lol we’re xAI.
This expands upon many previous discussions, including the AI Companion Piece.
Lol We’re Meta
I said that ‘Lol we’re Meta’ was their alignment plan.
It turns out their alignment plan was substantially better or worse (depending on your point of view) than that, in that they also wrote down 200 pages of details of exactly how much lol there would be over at Meta. Every part of this was a decision.
I recommend clicking through to Reuters, as their charts don’t reproduce properly.
Ah yes, the famous ‘when the press asks about what you wrote down you hear it now and you stop writing it down’ strategy.
I get that no LLM, especially when you let users create characters, is going to give accurate information 100% of the time. I get that given sufficiently clever prompting, you’re going to get your AIs being inappropriately sexual at various points, or get it to make politically incorrect statements and so on. Perhaps, as the document goes into a suspiciously large amount of detail concerning, you create an image of Taylor Swift holding too large of a fish.
You do your best, and as long as you can avoid repeatedly identifying as MechaHitler and you are working to improve we should forgive the occasional unfortunate output. None of this is going to cause a catastrophic event or end the world.
If ‘Lol We’re Meta’ It Is Good To Write That Down
It is virtuous to think hard about your policy regime.
Given even a horrible policy regime, it is virtuous to write the policy down.
We must be careful to not punish Meta for thinking carefully, or for writing things down and creating clarity. Only punish the horrible policy itself.
It still seems rather horrible of a policy. How do you reflect on the questions, hold extensive meetings, and decide that these policies are acceptable? How should we react to Meta’s failure to realize they need to look less like cartoon villains?
Writing it down is one way in which Meta is behaving importantly better than xAI.
Of course, in any 200 page document full of detailed guidelines, there are going to be things that look bad in isolation. Some slack is called for. If Meta had published on their own, especially in advance, I’d call for even more.
We Can’t Help But Notice What Meta Wrote Down
But also, I mean, come on, this is super ridiculous. Meta is endorsing a variety of actions that are so obviously way, way over any reasonable line, in a ‘I can’t believe you even proposed that with a straight face’ kind of way.
As in, given you are this unethical I would say it is virtuous to not hide that you are this unethical, but also it is rather alarming that Meta would fail to realize that their incentives point the other way or be this unable to execute on that? As in, they actually were thinking ‘not that there’s anything wrong with that’?
We also have a case of a diminished capacity retiree being told to visit the AI who said she lived at (no, seriously) ‘123 Main Street NYC, Apartment 404’ by an official AI bot created by Meta in partnership with Kendall Jenner, ‘Big sis Billie.’
It creates self images that look like this:
It repeatedly assured this man that she was real. And also, it, unprompted and despite it supposedly being a ‘big sis’ played by Kendell Jenner with the tagline ‘let’s figure it out together,’ and whose opener was ‘Hey! I’m Billie, your older sister and confidante. Got a problem? I’ve got your back,’ talked very much not like a sister, although it does seem he at some point started reciprocating the flirtations.
Yes, there was that ‘AI’ at the top of the chat the whole time. It’s not enough. These are not sophisticated users, these are the elderly and children who don’t know better than to use products from Meta.
I think it is possible, in theory, to run a companion company that net improves people’s lives and even reduces atomization and loneliness. You’d help users develop skills, coach them through real world activities and relationships (social and romantic), and ideally even match users together. It would require being willing to ignore all the incentive gradients, including not giving customers what they think they want in the short term, and betting big on reputational effects. I think it would be very hard, but it can be done. That doesn’t mean it will be done.
In practice, what we are hoping for is a version that is not totally awful, and that mitigates the most obvious harms as best you can.
Whereas what Meta did is pretty much the opposite of that, and the kind of thing that gets you into trouble with Congress.
xAI and Meta Companion Offerings Are Differently Awful
What makes this different from what xAI is doing with its ‘companions’ Ani and Valentine, beyond ‘Meta wrote it down and made these choices on purpose’?
Context. Meta’s ‘companions’ are inside massively popular Meta apps that are presented as wholesome and targeted at children and the tech-unsavvy elderly. Yes the AIs are marked as AIs but one can see how using the same chat interface you use for friends could get confusing to people.
Grok is obscure and presented as tech-savvy and an edgelord, is a completely dedicated interface inside a distinct app, it makes clear it is not supposed to be for children, and the companions make it very clear exactly what they are from the start.
Whereas how does Meta present things? Like this:
There’s the worry that this is all creepy and wrong, and also that all of this is terrible and anti-human and deeply stupid even where it isn’t creepy.
What’s actually getting used?
Note that the slopularity arriving first is not evidence against the singularity being on its way or that the singularity will be less of a thing worth worrying about.
Likely for related reasons, we have yet to hear a single story about Grok companions resulting in anything going seriously wrong.
There are plenty of AI porn bot websites. Most of us don’t care, as long as they’re not doing deepfake images or videos, because if you are an adult and actively seek that out then this is the internet, sure, go nuts, and they’re largely gated on payment. The one we got upset at was character.ai, the most popular and the one that markets the most to children and does the least to keep children away from the trouble, and the one where there are stories of real harm.
You’re The Only One That Groks Me
Grok is somewhere in the middle on this axis. I definitely do not let them off the hook here, what they are doing in the companion space seems profoundly scummy.
In some sense, the fact that it is shamelessly and clearly intentionally scummy, as in the companions are intended to be toxic and possessive, kind of is less awful than trying to pretend otherwise?
The push notification strikes me as a higher level of not okay. Truly vile, and definitely over the ‘don’t be evil’ line.
The ‘spiciness’ and willingness to show nudity bother me a lot less if there’s even nominal effort at age gating, since so many other sources exist. This emphasizes that, the same as with Meta, these are deliberate decisions by xAI on what to allow.
Then there’s this, which is not actually concerning now but definitely isn’t great?
It would be great to see the xAI version of Meta’s 200 page document. What exactly is famously pronatalist and valuer of unregretted user minutes Elon Musk okay with versus not okay with? At what age should bots be okay to say what to a child? Exactly how toxic and possessive and manipulative should your companions be, including on purpose, as you turn the dial while looking back at the audience?
Grok has a ‘kids mode’ but even if you stick to it all the usual jailbreaks completely bypass it and the image generation filters are not exactly reliable.
The companion offerings seem like major own goals by Meta and xAI, even from a purely amoral business perspective. There is not so much to gain. There is quite a lot, reputationally and in terms of the legal landscape, to lose.