Synopsis: I claim to describe the exact mental structure that allows qualia.
Background: What Is Consciousness?
Dehaene's Consciousness and the Brain rigorously differentiates conscious and unconscious activity. Consciousness, the book suggests, is correlated with events where the brain gathers all of its probability distributions about the world, and samples from them to build a consistent unitary world-model, on which it then acts. The experiments show that this is necessary for multi-step calculation, abstract thinking, and reasoning over agglomerations of distant sensory inputs.
However, the book's definition of consciousness is not a synonym for self-awareness. Rather, I would term the phenomenon it picks out as "moments of agency": as the state in which the mind can devise goal-oriented plans using a well-formed world-model, and engage in proper consequenialist reasoning. Outside those moments, it's just a bundle of automatic heuristics.
Self-awareness, I suspect, is part of these moments-of-agency in humans, but isn't the same thing generally. Just having Dehaene!consciousness isn't a sufficient condition for self-awareness: there's something on top of that going on.
What Is Self-Awareness?
What do we expect to happen to an agent the moment it attains consciousness, in the sense of perceiving itself to have qualia?
Why, it would start perceiving qualia — the keyword being perceive. It would start acting like it receives some sort of feedback from a novel sense, not unlike sight. Getting data about how it's like to be a thing like itself.
Let's suppose that it's nothing magical — that there isn't a species of etheric parasites which attach themselves to any sufficiently advanced engine of cognition and start making it hallucinate. Neither are qualia "emergent" — as if, if you formally wrote out an algorithm for general reasoning, that algorithm would spontaneously rewrite itself to be having these imaginary experiences. If self-awareness is as mundane as any other sense, then what internal mechanism would we expect to correspond to it?
When we "see", what happens is: some sort of ground-truth data enter a specialized sensory organ, that organ transmits the information to the brain, the brain parses it, and offers it to our conscious inspection, so we may account for it in planning.
If we view qualia as sense-data, it follows that they'd be processed along a similar pathway.
- What are the ground-truth data corresponding to qualia? The current internal state of this agent. The inputs it's processing, the configuration its world-model is in, its working-memory cache, the setting it's operating in, the suite of currently active processes...
- What is the specialized sensory organ, and how does it communicate with the brain? Despite what may seem, we do need one. The ground-truth state of the brain isn't by default "known" by the brain itself; it's just in that state. A specialized mechanism needs to know how to summarize raw brain-states into reports, then pool them together with other information about the world.
- How are the qualia-data interpreted? Much like visual information, they're parsed as the snapshot of all information received by the sensory organ at a particular moment. A self-model; a summary of how it's like to be you, perceiving what you perceive and feeling what you feel.
- (In theory, that should cause infinite recursion. A faithful self-model also has a self-model: you can consider what it's like to be someone who experiences being a someone. But seeing as we're bounded agents, I assume that algorithm is lazy.)
Let's make the distinction sharper. An agent gets hurt, information about that travels to the brain, where it's interpreted as "pain". Pain has the following effects:
- It updates the inner planner away from plans that cause the agent harm, like an NN getting high loss.
- It changes the current plan-making regime: the planner is incentivized to make plans in a hurried manner, and to consider more extreme options, so as to get out of the dangerous situation faster.
Which parts of that correspond to pain-qualia?
Pain-qualia are not pain inflicting changes upon a planner: by themselves, these changes are introduced outside the planner's purview. Consider taking an inert neural network, manually rewriting its weights, then running it. It couldn't have possibly "felt" that change, except by magic; it was simply changed. For a change to be felt, you need an outer loop that'd record your actions, then present the records to the NN.
That's what pain-qualia are: summaries of the effects pain has upon the planner that are fed as input to that very planner.
That leaves one question:
How Is It Useful?
Well, self-awareness evolved to be offered as an input to the planner-agent, so it must be used by the planner-agent somehow. The obvious answer seems correct here: meta-planning.
First, self-awareness allows an agent to account for altered states of consciousness. If it knows it's deliriously happy, or sad, or drugged, it'll know that it's biased towards certain actions or plans over others, and that these situational biases may not be desirable. So it'll know to correct its behavior to suit. (Note that mere awareness of an altered state is insufficient: feedback needs to be detailed enough to allow that course-correction.)
Second, it allows it to predict in detail its future plan-making instances. What, given a plan, its future selves would want to do at various stages of that plan, and how capable they'll be of doing this.
Concretely, self-awareness is what allows to:
- Know not to lash out while angry, even if it feels right and sensible in the moment, because you know your plan-making process is compromised.
- Know that a plan which hinges on your ability to solve highly challenging mathematical problems while getting your arm chopped off is a doomed one.
- Know not to commit to an exciting-seeming project too soon, because you know from past experience that your interest will wane.
To be clear, those are just examples; what we want is the ability to display such meta-planning universally. We can imagine an animal that instinctively shies away from certain drastic actions while angry, but the actual requirement is the ability to do that in off-distribution contexts in a zero-shot regime.
On that note... I'll abstain from strong statements on whether various animals actually have self-models complex enough to be morally relevant. I suspect, however, that almost no-one's planning algorithms are advanced enough to make good use of qualia — and evolution would not grant them senses they can't use. In particular, this capability implies high trust placed by evolution in the planner-part: that sometimes it may know better than the built-in instincts, and should have the ability to plan around them.
But I'm pushing back against this sort of argument. As I've described, a mind in pain does not necessarily experience that pain. The capacity to have qualia of pain corresponds to a specific mental process where the effect of pain on the agent is picked up by a specialized "sensory apparatus" and re-fed as input to the planning module within that agent. This, on a very concrete level, is what having internal experience means. Just track the information flows!
And it's entirely possible for a mind to simply lack that sensory apparatus.
As such, in terms of empirical tests for sentience, the thing to look for isn't whether something looks like it experiences emotions. It's whether, while plan-making, that agent can reason about its own behavior in different emotional states.
1. Cogito, Ergo Sum. It's easy to formalize. As per Dehaene, all of the inferences the brain makes about the outside world are probabilistic. When presented to the planner, they would be appropriately tagged with their probability estimates. The one exception would be information about the brain's own continued functioning: it would be tagged "confidence 1". After all, the only way for the self-awareness mechanism to become compromised involves severe damage to the brain's internals, which is probably fatal. So evolution never had cause to program us to doubt it.
And that's why we go around slipping into solipsism.
2. "Here's a simple program that tracks its own state. Is it sentient?" No. It needs to be an agent.
3. The Hard Problem of Consciousness. None of the above seems to address the real question: why does self-awareness seem so... metaphysically different from the rest of the universe? Or, phrased more tractably: "Why does a mind that implements the self-awareness mechanism start viewing self-aware processes as being qualitatively different, compared to other matter? And gets so confused about it?"
I'm afraid I don't have a complete answer to that, as I'm having some trouble staring at the thing myself. I feel confident, though, that whatever it is, it wouldn't invalidate anything I wrote above. I suspect it's a combination of two things:
- Cogito, ergo sum. Our existence feels qualitatively different because it's the only thing to which the mind assigns absolute confidence.
- A quirk of our conceptual vocabulary. It's not that self-aware things have some metaphysically special component, it's that there's a concept native to our minds that sharply differentiates them, so things we imagine as sentient feel qualitatively different to us.
Try to summon up, in the simulation spaces of your mind, a bunch of meaningless particles and a qualia-having thing. Contrast them. If I've successfully dissolved the concept of self-awareness for you, that "qualitative difference" might now feel as something truly indescribable. It's not even self-awareness now, it's— You can't make any assertions about it, except that it is! Whatever it is has to be part of the physical universe for you to reason about it — yet it's also completely, categorically different!
That should give you a pause. What does that even mean?
It's a trick of the mind. The difference you're looking at, I suspect, is a mentally primitive signifier for "has inherent moral relevance". It's fully internal to our minds, and is non-reducible in our conceptual vocabulary. And the "first-person view" is just a combination of that and absolute confidence being assigned to self-existence.
The whole thing's just a giant red herring.
4. "Does any of that have drastic moral implications?" No. It all adds up to normality. Self-aware things are of moral relevance, random rocks aren't. The Hard Problem being empty, if anything, only adds to that. With self-awareness and qualia thus dissolved, there isn't any additional "magical fluid" beyond them that can confer moral relevance to a bundle of matter.
So we can just go on valuing the self-aware agenty matter-bundles we've always valued. There's no room for any shocking twists where puddles turn out to have been important this entire time.
In the Dehaene sense: i. e., the data are pooled together and consolidated into a world-model which is fed as input to the planning algorithm.
Unless it really are etheric parasites or something. I mean, it might be!
And if you try for a bit, you can imagine an entity that should by all rights have it — a p-zombie — that inexplicably doesn't. I suspect the native use for this, and why "moral relevance" is independent of any local physical facts about a system at all, is to dismiss the lives of of enemy tribesmen as worthless.