dynomight's Shortform

dynomight

This is a special post for quick takes by dynomight. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

Just had this totally non-dystopian conversation:

"...So for other users, I spent a few hours helping [LLM] understand why it was wrong about tariffs."

"Noooo! That does not work."

"Relax, it thanked me and stated it was changing its answer."

"It's lying!"

"No, it just confirmed that it's not lying."

yeah I've seen people say things like that as well - "I confirmed" used by a human to refer to asking an LLM to state what is true. I haven't figured out how to get a person who is using that sentence to engage with the fact of the matter reliably, but I did have success one time, by explaining in detail how an AI actually works. I would love love love to be put in contact with the person you're describing here, and especially if they'd be willing to share the conversation they had; the reason I'd give them is probably "so that other AIs can also learn fro... (read more)

3ZankerH1y

Is there a meaningful difference between what you mean by "convinced" in this context and "triggered sycophancy mode"? If so, what is it?

2Trinley Goldenberg1y

It doesn't seem like either thing captures the gears of what's happening to an llm, but it's insofar as you're arguing it's impossible for an llm to be convinced, I think you should provide better gears or what specific aspect of "convincing" you don't think applies I dint think that arguing with an llm will reliably enter into a sychophant mode, or that the thing that happens when you go through a series of back and forth argumentation with an llm can be fully explained by it entering into a mask/face

dynomight's Shortform

7

dynomight's Shortform

7