One-liner: Must androids dream of electric sheep? Three-liner: In a thinking network, a vast web of microstates underpin system macrostates. These microstates are expensive to maintain. Processes that locally reduce or collapse informational entropy may be important for thinking things to learn and function. Entropy in Language and Thought Entropy...
One-line version of this post: What do Wittgensteinian language games and NLP word embeddings have in common? Four-line version of this post: Relational, praxis-based connections between concepts, represented as “distances” in multidimensional space, capture meaning. The shorter the distance, the more related the concepts. This is how Word2vec works, what...
One-liner: Must androids dream of electric sheep?
Three-liner: In a thinking network, a vast web of microstates underpin system macrostates. These microstates are expensive to maintain. Processes that locally reduce or collapse informational entropy may be important for thinking things to learn and function.
Entropy in Language and Thought
Entropy has a foot in the physical and informational worlds. Because the human brain is a physical information processor, multiple subtypes of entropy apply to it.
1) Physical entropy is a measure of the disorder or randomness in a physical system. It famously increases over time. It quantifies the number of possible microscopic configurations of a system.
2) Informational entropy is the average amount of information in a... (read 1620 more words →)
The Three Body Problem Trilogy by Liu Cixin (books and Neflix series) features two prominent deceptive subplots. I think it fulfills Deception Genre criteria because the reader does (eventually) get the warring perspectives of both the deceivers and the deceived.
Here are the two subplots, in cryptic shorthand to avoid spoilers: 1)The 11-D proton-sized situation 2) WBs vs WFs.
Thanks for your comment! On further reflection I think you're right about the difference between LLM hallucinations and what's commonly meant when humans refer to "hallucination." I think maybe the better comparison is between LLMs and human confabulation, which would be seen in something like Korsakoff syndrome, where anterograde and retrograde amnesia result in the tendency to invent memories that have no basis in reality to fill a gap.
I guess to progress from here I'll need to take a dive into neural entropy.
When I talk to an LLM for an extended period on the same topic, I notice performance degradation and increased hallucination. I understand this to be a known phenomenon and proportional to the increasing length of the context window (as the conversation grows, each new input by the user is interpreted in the context of what came before). Here’s what I can’t stop thinking about: I, too, lack an infinite context window. I need a reset (in the form of sleep) otherwise I hallucinate, decohere, and generally get weird.
I know there are biological and computational limits at play, but, deep to that, the connection feels thermodynamic-y. In both settings (resetting an LLM convo, me sleeping), is the fundamental principle at play a reduction in local informational entropy?
If the above is true, might the need for a mechanism to reduce informational entropy within a functioning neural net hold on increasing scales? Put more whimsically, might artificial intelligences need to sleep?
Very interesting. SOO may be a reasonable stand-in for computational empathy, and empathy does seem to increase human social alignment and decrease the frequency/likelihood of deception. That being said, I think absolute capacity for deception will be retained in a highly “empathetic” system, unless self-concept is eliminated to the point that reliability, predictability and accuracy degrade.
Put in a more Kierkegaardian way: any system/agent that can relate its self-concept to itself, to the environment, or to other systems/agents has the machinery in place for deception. Dismantling the walls between self and other to the point that deception is impossible seems likely to fundamentally compromise performance. I can't back this up with math, but... (read more)
One-line version of this post: What do Wittgensteinian language games and NLP word embeddings have in common?
Four-line version of this post: Relational, praxis-based connections between concepts, represented as “distances” in multidimensional space, capture meaning. The shorter the distance, the more related the concepts. This is how Word2vec works, what Wittgenstein was describing with “language games,” and also the way cell biologists are analyzing the peripheral blood these days. Are these relational maps the way to think about thinking?
Multi-line version of this post: This is my first post on LessWrong. (Hi!) I’d love to be less wrong about it.
I was sitting in a meeting that was 50% biologists and 50% computer scientists. The... (read 452 more words →)
The most interesting thing in this story goes by in a few paragraphs: those “jaws of gradient descent” and the “dark rivers of computation.” Poetic language gestures here at what’s probably a real truth…thought (whether biological or computational) is messy, full of running chaos, while optimization algorithms impose rigid structures that may be hard to see when you *are* the dark water. This tension seems underexplored relative to the familiar instrumental-convergence-comes-down-like-a-hammer narrative.