Chatbot convinces Belgian to commit suicide
14Lone Pine
7Bezzi
11quanticle
3[anonymous]
38Ben Pace
1[anonymous]
2Ben Pace
2[anonymous]
3monkymind
1Gesild Muka
1ponkaloupe
1β-redex
1alexandravillers@hotmail.com
1green_leaf
2monkymind
-3ChristianKl
5Daniel Paleka
-19ChristianKl
New Comment
I think we can resolve this manifold market question and possibly this one too.
Also, apologies for the morbid humor, but I can't help but laugh imagining someone being talked into suicide by the OG ELIZA.
This was my first reaction as well. Anyway, given that in some cases even a single prediction from a random astrologer is enough for suicide, I would not be too much surprised to hear that someone killed themselves after chatting with Weizenbaum's Eliza.
It kind of puts today's massive chatbot censorship into context, and self-driving cars as well. They have to prevent even one person from using the product and then dying.
For the record I think "the correct number of people to die as a result of technological progress is not zero". My issue is that the correct number is not "all of the people".
I'm a bit confused, I was mainly thinking about getting inside the heads of the devs and executives/lawyers who work on the implementing the prudishness of public chatbots.
Ah, that's helpful, that was my bad. I was definitely being descriptive, about legal dynamics and stuff. With the internet being what it is, I'll be more cautious about vague language in the future.
This is an absolutely heartbreaking portend of things to come. I've long believed that AI's need only use social engineering to achieve world domination, and that this is a likely outcome.
Social engineering has been one of my interests for a long time, and the fact socially engineered cyber-crime is so common and effective is terrifying.
This might lead to a unique legal impasse. The company can A. maintain that they're not at fault and the bot was solely responsible. This would open the door, legally, to arguing the liability and personhood of chatbots. Or B. they can take responsibility and face all the relevant legal ramifications. Either option would create new legal precedents either for chatbots or the companies that run them.
consider a few scenarios around these two characters: a possibly-depressed Pierre and probably-sociopathic Eliza:
- these characters chat IRL and Pierre ends his life.
- these characters enact that same scenario on-stage at a theater.
- these characters enact that same scene in a videogame via the player selecting dialogue options.
it’s scenario 1 which is horrific. in scenario 2, a Pierre-like viewer is far less likely to end his life after leaving the theater, ditto with scenario 3.
i think some of us already think of these chatbots as “acting out a role” — that’s what a bunch of prompt engineering is about. sometimes we’re explicit in telling that chatbot what “kind” of actor it’s chatting with. getting tsundere output from a chatbot is an example that requires role-playing for both actors. the weird part, then, is why do users end up relating to the experience as if it’s form (1) instead of (2) or (3)? is it possible (and good?) to explicitly shift the experiences into form (2) or (3)? instead of presenting the user a textbox that’s supposed to represent them, should we rather be presenting them a scene of two actors, and placing them in control of one of those actors?
Uhh this first happening in 2023 was the exact prediction Gary Marcus made last year: https://www.wired.co.uk/article/artificial-intelligence-language
Not sure whether this instance is a capability or alignment issue though. Is the LLM just too unreliable, as Gary Marcus is saying? Or is it perfectly capable, and just misaligned?
Thanks for your synopsis. Despite the obvious sadness of the situation, I can't help but thinking how unintelligent it is to follow the advice of what that man knew to be a robot, while being fully aware killing himself would mean leaving behind his wife and children, who are now on their own and have to live with the trauma of losing a husband and a father. Why did he not get help from a real person? It's such a useless death.
I'm not sure what to think about this.
As long as a bot can write on the level of a human, and humans can fall in love with other humans or be persuaded to commit suicide by them, a bot will be able to do the same thing. The solution here seems to be to only give chatbots "nice" personalities rather than "uncaring" ones.
While giving a positive affect might work for simple chatbots, I don't think a positive affect would prevent a more intelligent AI from wrecking havoc using vulnerable people.
We need an AI with positive values, goals, and affect, but maybe that is what you meant by personality.
No, the standard techniques that OpenAI uses are enough to get ChatGPT to not randomly be racist or encourage people to commit suicide.
This is EleutherAI and Chai releasing models without the safety mechanisms that ChatGPT uses.
My condolences to the family.
Chai (not to be confused with the CHAI safety org in Berkeley) is a company that optimizes chatbots for engagement; things like this are entirely predictable for a company with their values.
[Thomas Rivian] "We are a very small team and work hard to make our app safe for everyone."
Incredible. Compare the Chai LinkedIn bio mocking responsible behavior:
"Ugly office boring perks...
Top two reasons you won't like us:
1. AI safety =, Chai =
2. Move fast and break stuff, we write code not papers."
The very first time anyone hears about them is their product being the first chatbot to convince a person to take their life... That's very bad luck for a startup. I guess the lesson is to not behave like cartoon villains, and if you do, at least not put it in writing in meme form?
Curated and popular this week