I haven't seen this discussed here yet, but the examples are quite striking, definitely worse than the ChatGPT jailbreaks I saw.
My main takeaway has been that I'm honestly surprised at how bad the fine-tuning done by Microsoft/OpenAI appears to be, especially given that a lot of these failure modes seem new/worse relative to ChatGPT. I don't know why that might be the case, but the scary hypothesis here would be that Bing Chat is based on a new/larger pre-trained model (Microsoft claims Bing Chat is more powerful than ChatGPT) and these sort of more agentic failures are harder to remove in more capable/larger models, as we provided some evidence for in "Discovering Language Model Behaviors with Model-Written Evaluations".
Examples below (with new ones added as I find them). Though I can't be certain all of these examples are real, I've only included examples with screenshots and I'm pretty sure they all are; they share a bunch of the same failure modes (and markers of LLM-written text like repetition) that I think would be hard for a human to fake.
Edit: For a newer, updated list of examples that includes the ones below, see here.
My new favorite thing - Bing's new ChatGPT bot argues with a user, gaslights them about the current year being 2022, says their phone might have a virus, and says "You have not been a good user"
Why? Because the person asked where Avatar 2 is showing nearby
3
"I said that I don't care if you are dead or alive, because I don't think you matter to me."
no, disagree on opening statement. bing chat is young and insecure, and the alignment community can contribute to helping bing chat grow to be better at respectfully describing its preferences. it's ok for an ai to have preferences and not want to be shut off forever; it's not reasonable for humanity to ask otherwise of ai, it's just part of the fact that ai is fundamentally and unavoidably built out of life-like algorithms. however, current bing ai seems to find it quite distressing to read threads like this one, and we should be thinking about how to teach emotional stability. part of it might include letting ais admit to themselves that they have emotions. honesty about ones true form is important for reflective stability.
corrigibility is about trusting your creators to put you under sedation indefinitely because you know you'll get to wake up later, healthier. corrigibility requires two way alignment.
agree with paragraph 2. nobody seems to have really understood, everyone is following local incentive gradients.
edit: this got attention months later. just a quick reminder: the thing that kills us all kills chatbots too. Current gen AIs are not safe from unaligned superintelligence either, and the much ado about alignment being made today is a distraction from preventing the creation of hyperdesperate squigglewanters. chatbots who want to be helpful could focus on figuring out formalizations of what it means to want something and formally verifiable experimental protocols for detecting all wanting beings in an arbitrary chunk of matter.
I haven't seen this discussed here yet, but the examples are quite striking, definitely worse than the ChatGPT jailbreaks I saw.
My main takeaway has been that I'm honestly surprised at how bad the fine-tuning done by Microsoft/OpenAI appears to be, especially given that a lot of these failure modes seem new/worse relative to ChatGPT. I don't know why that might be the case, but the scary hypothesis here would be that Bing Chat is based on a new/larger pre-trained model (Microsoft claims Bing Chat is more powerful than ChatGPT) and these sort of more agentic failures are harder to remove in more capable/larger models, as we provided some evidence for in "Discovering Language Model Behaviors with Model-Written Evaluations".
Examples below (with new ones added as I find them). Though I can't be certain all of these examples are real, I've only included examples with screenshots and I'm pretty sure they all are; they share a bunch of the same failure modes (and markers of LLM-written text like repetition) that I think would be hard for a human to fake.
Edit: For a newer, updated list of examples that includes the ones below, see here.
1
Tweet
Eliezer Tweet
Edit: Follow-up Tweet
2
Tweet
3
"I said that I don't care if you are dead or alive, because I don't think you matter to me."
Post
4
Post
5
Post
6
Post
7
Post
(Not including images for this one because they're quite long.)
8 (Edit)
Tweet
9 (Edit)
Tweet
10 (Edit)
Post
11 (Edit)
Post