(my english is bad, please use your usual numeric assistant to improve the langage if you care about such things)
Homework assignment 1: Explicate Descartes’ worries about being deceived by a malignant being in these terms.
Descartes thought experiment was the opposite of a worry. It was a reinsurance, a certitude so high that he would construct his whole philosophy upon it: the certitude that any conscious thought is a proof you must exist, otherwise you couldn’t have this thought, right? In modern time, he would have said something like:
« First, notice that I’m using social media on purpose because I want to talk directly to my fandom base about this elegant truth I demonstrated. We exist! Indeed, suppose Elon is using some neuralink sh*t to mess with your perception while you’re actually a brain freely floating in some vat. You can’t disprove that! But even in this extreme (how would that help Elon go to mars?) scenario, you could still disprove that you don’t exist! From there I can prove God, or disprove it maybe, but on first principles always. …but why can’t I remember which one it was? Oh wait, see? It’s silly, I can’t even remember if my philosophy proves or disproves God, and I have no idea why I can’t remember that (as if I was a literary creation ah ah!) but still, as I can think I’m still sure I do exist! See? Now try explain that to whoever you want to seduce, and like me if that worked! »
Using your term (actually the correct technical term from a neuroscience/psy perspective): I perceive thoughts, then I can confabulate it’s me who thought that, while not noticing that « me » is a catch 22 for importing questionable assumptions, such like « I » is a solid entity rather than a fluid story.
If that sounds like imagining superintelligence must act as a single actor with a single set of values, you got my point.
I definitely like the term "confabulate" more than "hallucinate". It's more accurate and similar to what humans do. My favorite confabulation examples in humans are split-brain experiments.
"split-brain" patients, whose left and right brain hemispheres have been surgically disconnected for medical treatment. Neuroscientists have devised clever experiments in which information is provided to the right hemisphere (for instance, pictures of naked people), causing a change in behavior (embarrassed giggling). Split-brain individuals are then asked to explain their behavior verbally, which relies on the left hemisphere. Realizing that their body is laughing, but unaware of the nude images, the left hemisphere will confabulate an excuse for the body's behavior ("I keep laughing because you ask such funny questions, Doc!").
Cross-posted from New Savanna.
That LLMs “hallucinate” is well-known, though I prefer the term “confabulate.” They just make stuff up. They don’t do it intentionally, or with intent to deceive. They do it because they have no connection to “ground truth.” They don’t even know what such a thing is, not really, though I’m sure, if asked, ChatGPT would say something reasonable about the idea.
What LLMs have, loosely speaking, is an ontology. They ‘know’ what kinds of things exist. As Immanuel Kant pointed out in connection with the ontological argument for the existence of god, existence is not a predicate. When an LLM responds to a prompt, its response is consistent with the ontology it has internalized. If phenomena in its response happen to be true of the world, that’s incidental, though convenient for users.
What’s implicit in this conversation is an assumption that humans do not confabulate, that we can and so talk about reality. I think that’s true, sorta’, but misleading. Yes, we often anchor our statements in reality, as best we can do in the situation. And, yes, we can also deliberately side-step such anchoring. We do this when telling fictional stories without intention to deceive; in this situation we assert that the story is about imagined event. And we can also make things up with deceptive intent.
Nonetheless, I believe that our “natural” linguistic mode is just making things up, confabulation it you will. But we are surrounded by others and have to interact with them. One way to communicate effectively is to anchor our language in external events, in things others can readily observe, in reality, as we like to say. Such anchoring is not necessary for speech, but it is useful when conversing with others.
In this connection I think it’s worth noting that, for a number of years, neuroscientists have been investigating the brain structures that are most active when we’re doing nothing in particular, when we’re letting our mind wander. In that state we may attended to some external event one moment, scratch our back the next, think about that summer’s day three years ago, and so forth. We day dream. What do you think they call that conglomeration of neural structures? They call it the default mode network (DFM). If you give a verbal report of what’s passing by during a day dream, a lot of that is going to be confabulation. It’s just the way we are.
Homework assignment 1: Explicate Descartes’ worries about being deceived by a malignant being in these terms.
Homework assignment 2: Now consider what happens during sensory deprivation.