Total non-expert on LLM training here (on the details, anyway). My recent thinking about alignment (as a technical, mechanistic step in producing a finished LLM product) is that it "feels" to me akin to the "schooling" (right/wrong instructional teaching) of humans. It produces mixed results. One of the main quirks in my mind is the funny observation that behavior will have a surface and a subtext intent (trying to please the master enough to be left alone, and keeping one's job, while also sticking it to the man behind his back).
So, for me, the Goodhart r...
I don't comment a lot, but I felt this one was definitely worth the read and my time.
While I don't necessarily agree with every aspect, much of this resonated with how I see social media has (been) warped from a regular market of social connection to a lemon market, where the connection is crappy, and many sane people I know are blinding themselves to it (leaving in some corners behind a cesspool of the dopamine hit addicted).
Ultimately, this also seems to be true about how people have responded to the latest wave of human-rights initiatives (DEI) carried ...
My strong hunch is that this is true for almost any form of communication (internal and external) we receive: it conveys something we can extract value from if we are able to look past the surface (propositional content) of what we immediately infer.
And how difficult it is to remain open to the possibility that my first impression of a signal is "incorrect" (I got it wrong on my first attempt), given how frequently I have used my inference (first impression), and I am still alive (adaptive value of my past choice to not question my first impression)...
The ...
Some thoughts that came up for me while reading this piece (THANK YOU for putting this all together!!):
I suspect that the principles you describe around the "experience of tanha" go well beyond human or even mammalian psychology. If I am not mistaken, they arise out of a failure to appropriately incorporate the non-life-matter sort of conflicts (between elements of a whole and the whole) as part of life. The cells in my body all have different "cultures" (needs of chemical milieus, whether or not bacteria are needed for the "gut" process, or are absolutely...
Not too sure if I misunderstood the initial premise of the post, but the examples given seem not to be "similar" enough for me to follow the reasoning about the cause(s) for the dialog going poorly to be the same.
The one commonality I can immediately make out is that the initial proposition seems too unspecific for the interlocutor to know what the intention of the speaker actually is (other than to answer with, "you're exactly right, so now what?").
At the very least, if I add an implied question of, "assuming you agree with P, what do you suggest we do ab... (read more)