My theory is that the LLM knows what is persuasive and brute forces it. So we see a ton of repetition (which ironically, is a costly transformer signal, as if writing with ink and pen), and we see antithesis. Regarding other persuasive methods (alliteration etc.), more research is needed. Seems like it is more and more fine tuned to what people 'like', possibly by a feedback loop.
What does it mean? It means that we writers are under-using persuasive brute-force methods. We write to sound natural and not specifically persuasive, though in general we do want to be persuasive; this comes from a very human instinct of seeming 'as if' "not trying"; LLM don't have that self-conscious game. It finds the methods, and then uses them ad-nauseum because it works. Why do we have this self-conscious such that we become hyper aware of AI persuasive use?
It works for getting the typical LMArena user to click the like button, but it's not clear that it works for persuasion or anything else. Personally I find the style very offputting and usually stop reading when I notice it.
I think it might be that the undesired response in RLHF/DPO settings isn't good enough.
Imagine two responses, one leveraging stylistic devices and persuasive words while the other... well, just doesn't. Naturally the first is better and more desirable. If we now inspect this over the whole training batch, these distinctions of the preferred response at any point in the response leveraging stylistic devices will become clear. That is, a phrase like "It's not an X, it's a Y" will occur at a bunch of different positions throughout all the different positive examples in contrast to the negative examples which very rarely, if at all, showcase such pleasant phrasing.
But wouldn't then such behavior of constantly repeating stylistic devices be exactly what we would expect? This clear contrast between positive and negative example will be what we distill in our final model, basically telling it that stylistic devices at any point are preferred.
To move away from this, looking for better high quality positive examples won't be helpful at all - instead we need the negative examples throughout training to become closer and closer to the positive examples just like a writer progresses through his career: first learning about stylistic devices, then understanding when to use them meaningfully, when less is more and finally fully mastering it. This contrast between good and really good writer needs to be captured more in the posttraining data for something like DPO.
Do take this with a grain of salt, just a random theory i came up with while thinking about this for 10 minutes or so, i didn't research the empirical state of research with this hypothesis but it does seem somewhat convincing to me at least.
There are many different stylistic devices that humans writers use. I believe that there's a subset of stylistic devices that all LLMs use. Do you believe this isn't the case?
My guess is that they do so in imitation of humans who do the same thing when asked the sorts of questions that people ask LLMs. It's not an LLM thing; it's a thing one does to make distinctions clear, when the other person might otherwise conflate two distinct entities, clusters, or topics. It just so happens that people ask LLMs a lot of that sort of question, and thus elicit a lot of that particular response.
(I also use em dashes, yes.)
More specifically, they may be emulating the Kenyans who were hired to create much of the training data. "I'm Kenyan. I Don't Write Like ChatGPT. ChatGPT Writes Like Me."
Note: I can't verify that the post I linked is legitimate. For all I know it could be generated by ChatGPT instructed to emulate a Kenyan writing about ChatGPT. HN discussion here.
There seem to be common patterns of how LLMs write text that's shared by the LLMs of different companies and where the language patterns the LLM uses are different from usual human writing.
How much do we know about why LLMs pick certain patterns? Do we know why they use "It's not an X, it's a Y? If not, maybe understand why they pick patterns like it can make us better understand how LLMs are reasoning?