My own take on sentience/consciousness is somewhat different from yours, and in the general case, I think Anil Seth's theory is better than Global Workspace Theory, but Global Workspace Theory does explain the weird properties of consciousness in humans better.
For more, read this review:
https://www.lesswrong.com/posts/FQhtpHFiPacG3KrvD/seth-explains-consciousness#7ncCBPLcCwpRYdXuG
On this:
On that note... I'll abstain from strong statements on whether various animals actually have self-models complex enough to be morally relevant. I suspect, however, that almost no-one's planning algorithms are advanced enough to make good use of qualia — and evolution would not grant them senses they can't use. In particular, this capability implies high trust placed by evolution in the planner-part: that sometimes it may know better than the built-in instincts, and should have the ability to plan around them.
But I'm pushing back against this sort of argument. As I've described, a mind in pain does not necessarily experience that pain. The capacity to have qualia of pain corresponds to a specific mental process where the effect of pain on the agent is picked up by a specialized "sensory apparatus" and re-fed as input to the planning module within that agent. This, on a very concrete level, is what having internal experience means. Just track the information flows!
And it's entirely possible for a mind to simply lack that sensory apparatus.
I think that there doesn't need to be high trust placed in the planner part, because self-modeling is basically always useful if you want to stay alive, thanks to the gooder regulator theorem by John Wentworth, and self-modeling/reasoning is more like a continuum than sharply discrete.
On why the hard problem seems hard:
I'm not sure, but I suspect that it's a short-hand for "has inherent moral relevance". It's not tied to "is self-aware" because evolution wanted us to be able to dehumanize criminals and members of competing tribes: view them as beasts, soulless barbarians. So the concept is decoupled from its definition, which means we can imagine incoherent states where things that have what we define as "qualia" don't have qualia, and vice versa.
I think this is a likely answer, and that's despite the post below being bad methodologically for reasons Paradiddle and Sunwillrise says:
https://www.lesswrong.com/posts/KpD2fJa6zo8o2MBxg/consciousness-as-a-conflationary-alliance-term-for
Another reason why the hard problem seems hard is that way too many philosophers are disinclined to gather any data on the phenomenon of interest at all, because they don't have backgrounds in neuroscience, and instead want to purely define consciousness without reference to any empirical reality, which is a very bad approach to learning, and also self-report data is pretty terrible as data, unfortunately, when they do gather data, and they don't realize this.
I have a somewhat different takeaway for ethics.
In contrast to this:
4. Moral Implications. If you accept the above, then the whole "qualia" debacle is just a massive red herring caused by the idiosyncrasies of our mental architecture. What does that imply for ethics?
Well, that's simple: we just have to re-connect the free-floating "qualia" concept with the definition of qualia. We value things that have first-person experiences similar to ours. Hence, we have to isolate the algorithms that allow things to have first-person experiences like ours, then assign things like that moral relevance, and dismiss the moral relevance of everything else.
And with there not being some additional "magical fluid" that can confer moral relevance to a bundle of matter, we can rest assured there won't be any shocking twists where puddles turn out to have been important this entire time.
I instead argue that we need to decouple what we value (which can be arbitrary) from what things actually have qualia (which has a single answer in general), and I absolutely disagree with the claim that we have to dismiss the moral relevance of everything else, and I also disagree with the claim that we have to assign moral relevance to algorithms that allow things to have first-person experiences like ours.
It seems to be taken for granted here that self-awareness=qualia. If something is self-aware and talking or thinking about how it has qualia, that sure is evidence of it having qualia, but I'm not sure the reverse direction holds. What about internal-state-tracking is necessary for creating the mysterious redness of red exactly, or the hurt-iness of pain?
I can see how pain as defined above the spoiler section doesn't necessarily lead to pain qualia, and in many simple architectures obviously doesn't, but I don't see how processing a summary of pain effects on the network does lead to it. Say the summary is a single bit that's either 0, "no pain right now", or 1, "pain active". What makes that bit feel hurty to the network, instead of, say, looking red, or smelling sweet, or any other qualia? I don't feel any more able to answer these questions after adding the hypothesis "self-awareness necessary" to my model of the situation.
My mind sure agrees that it's kind of suspicious how these mysterious qualia thingies only ever seem to exert a direct influence on the world when agents engage in modelling and introspection about them, and maybe that's hinting that self-awareness is, or causes, qualia somehow. But I've never gotten further than this vague intuition in justifying or modelling the connection.
Great questions!
What about internal-state-tracking is necessary for creating the mysterious redness of red exactly, or the hurt-iness of pain?
Well, as you note, the only time we notice these things is when we self-model, and they otherwise have no causal effect on reality; a mind that doesn't self-reflect is not affected by them. So... that can only mean they only exist when we self-reflect.
Say the summary is a single bit that's either 0, "no pain right now", or 1, "pain active". What makes that bit feel hurty to the network, instead of, say, looking red, or smelling sweet, or any other qualia?
Mm, the summary-interpretation mechanism? Imagine if instead of an eye, you had a binary input, and the brain was hard-wired to parse "0" from this input as a dog picture, and "1" as a cat picture. So you perceive 1, the signal travels to the brain, enters the pre-processing machinery, that machinery retrieves the cat picture, and shoves it into the visual input of your planner-part, claiming it's what the binary organ perceives.
Similarly, the binary pain channel you're describing would retrieve some hard-coded idea of how "I'm in pain" is meant to feel, convert it into a format the planner can parse, put it into some specialized input channel, and the planner would make decisions based on that. This would, of course, not be the rich and varied and context-dependent sense of pain we have — it would be, well, binary, always feeling the same.
I want to respond to this, but I'm someone who cares intensely about animal rights and thus I have to work around my instinctive reaction of blind rage at someone who implies they might not be conscious, and try to communicate in a reasonable, respectful way.
My best counterargument to all this is the simple fact that humans have dreams. No one is self-aware in a sufficiently nonlucid dream. I have never once that I know of made a clear plan in a dream - I only ever react to what is happening around me. Nonetheless I experience qualia.
Also I wildly disagree with you that consciousness and self-awareness have anything to do with one another. The latter is an accident of evolution that humans have found quite useful, but it's not intrinsically special or necessary. In fact I don't even think self-awareness is morally relevant - only consciousness in general is - and I think that panpsychism is true - that is, qualia are universal and exist in every physical system which is doing any kind of computation, but they have degrees of complexity that differ, and ours happen to be rather complex.
All of these are my opinions though, and they are partly emotionally motivated; I'm not sure how to actually provide any kind of counterargument, other than the copernican principle: anything that implies humans are radically different from other animals in a qualitative way, rather than merely having crossed a quantitative threshold over which some abilities manifest that were only latent before, is frankly absurd.
Note, I'm not saying your analysis here doesn't have value - it's a great explanation of the kinds of qualia only self-aware beings have - but not qualia in themselves. I do not have to be able to perceive my own existence in order to perceive colors, sounds, and sensations, and people who are asleep or partly so, or otherwise have their self-awareness heavily modified, nonetheless experience qualia.
No one is self-aware in a sufficiently nonlucid dream.
No one is self-aware in a sufficiently nonlucid X, no matter what X is.
I want to respond to this, but I'm someone who cares intensely about animal rights and thus I have to work around my instinctive reaction of blind rage at someone who implies they might not be conscious, and try to communicate in a reasonable, respectful way.
I appreciate it.
No one is self-aware in a sufficiently nonlucid dream
And when a person closes their eyes, they stop being able to see. Their inner planner starts to ignore visual input, or might not make active plans at all if you're just taking a moment to rest. Does that mean that the machinery for vision disappears?
Dreams, or moments of intense pain as the post I'd linked to describes, are just the analogue for qualia. It keeps generating self-reports and keeps storing them in memory, but the planner simply doesn't use them. But when you wake up, and turn your attention to your dream-memories, they're there for you to look at.
I think that panpsychism is true - that is, qualia are universal and exist in every physical system which is doing any kind of computation
But what does that mean? When you imagine a non-self-aware entity perceiving colors, what do you imagine happening in its mind, algorithmically?
Dreams don't suddenly become real only after you wake up. That's utterly absurd. I know for a fact I am conscious while I am dreaming - it's not just a bunch of memories I can only access later. I'm frankly flabbergasted that you believe otherwise.
When I imagine a non-self-aware entity perceiving colors, I imagine that the part of their brain involved in sensory processing is... doing the sensory processing! There is something which it is like to have a temporal lobe analyzing data filtered through the retina, regardless of what else is in your brain! You can't just be doing complex computations like that without experiencing them.
And by the way, when I close my eyes, I can still see. There might not be much to look at, but my visual processing still works. That part of your response was badly worded.
Your "planner" seems like a confused, pseudo-physicalized version of a soul to me. You haven't actually solved the problem of how qualia arise in the first place, you've just shoved it into a corner of the brain and claimed to dissolve the problem. Okay, so suppose you're right and only the planner part of the brain has qualia, and qualia exist only because of it - but what are they? We both know there is, in fact a difference between qualia and basically everything else in the universe. You can't solve the hard problem by saying "aha, the brain doesn't do it, a specific part of the brain does it!"
Meanwhile, if you believe as I do that qualia are just what computations of all kinds "look like from the inside", the problem actually does dissolve. If you haven't looked into integrated information theory, I would suggest you do so: I think they are somehow incorrect as their theory makes a few unintuitive predictions, but it's the closest to what I think of as a reasonable theory of consciousness that I've ever seen.
I'd be interested to hear how this compares with Wolfgang Schwarz's ideas in 'Imaginary Foundations' and 'From Sensor Variables to Phenomenal Facts'. Sounds like there's some overlap, and Schwarz has a kind of explanation for why the hard problem might arise that you might be able to draw on.
Link to the second of the papers mentioned: https://www.umsu.de/papers/sensorfacts.pdf
There's some overlap indeed. In particular, it definitely aligns with my model of why the redness of red is "flavoured" as the redness of red and not as burning your fingers (because it's routed through the associations in our world-model). But it seems to jump too quickly to "having senses + WM implies qualia"; I think the self-awareness loop I'd described is still necessary on top of that.
No. It needs to be an agent.
Why? Also everything is an agent that successfully optimizes itself to be in the state it currently is. And what counts for an outer loop is arbitrary in the same way.
Builds up on: Consciousness and the Brain by Stanislas Dehaene. Good summaries may be found here or here, though reading them is not strictly necessary.
Synopsis: I claim to describe the exact mental structure that allows qualia.
Dehaene's Consciousness and the Brain rigorously differentiates conscious and unconscious activity. Consciousness, the book suggests, is correlated with events where the brain gathers all of its probability distributions about the world, and samples from them to build a consistent unitary world-model, on which it then acts. The experiments show that this is necessary for multi-step calculation, abstract thinking, and reasoning over agglomerations of distant sensory inputs.
However, the book's definition of consciousness is not a synonym for self-awareness. Rather, I would term the phenomenon it picks out as "moments of agency": as the state in which the mind can devise goal-oriented plans using a well-formed world-model, and engage in proper consequenialist reasoning. Outside those moments, it's just a bundle of automatic heuristics.
Self-awareness, I suspect, is part of these moments-of-agency in humans, but isn't the same thing generally. Just having Dehaene!consciousness isn't a sufficient condition for self-awareness: there's something on top of that going on.
What do we expect to happen to an agent the moment it attains self-awareness, in the sense of perceiving itself to have qualia?
Why, it would start perceiving qualia — the keyword being perceive. It would start acting like it receives some sort of feedback from a novel sense, not unlike sight. Getting data about how it's like to be a thing like itself.
Let's suppose that it's nothing magical — that there isn't a species of etheric parasites which attach themselves to any sufficiently advanced engine of cognition and start making it hallucinate. Neither are qualia "emergent" — as if, if you formally wrote out an algorithm for general reasoning, that algorithm would spontaneously rewrite itself to be having these imaginary experiences. If self-awareness is as mundane as any other sense, then what internal mechanism would we expect to correspond to it?
When we "see", what happens is: some sort of ground-truth data enter a specialized sensory organ, that organ transmits the information to the brain, the brain parses it, and offers it to our conscious inspection[1], so we may account for it in planning.
If we view qualia as sense-data, it follows that they'd be processed along a similar pathway.
Let's make the distinction sharper. An agent gets hurt, information about that travels to the brain, where it's interpreted as "pain". Pain has the following effects:
Which parts of that correspond to pain-qualia?
None.
Pain-qualia are not pain inflicting changes upon a planner: by themselves, these changes are introduced outside the planner's purview. Consider taking an inert neural network, manually rewriting its weights, then running it. It couldn't have possibly "felt" that change, except by magic; it was simply changed. For a change to be felt, you need an outer loop that'd record your actions, then present the records to the NN.
That's what pain-qualia are: summaries of the effects pain has upon the planner that are fed as input to that very planner.
That leaves one question:
Well, self-awareness evolved to be offered as an input to the planner-part, so it must be used by the planner-part somehow. The obvious answer seems correct here: meta-planning.
First, self-awareness allows an agent to account for altered states of consciousness. If it knows it's deliriously happy, or sad, or drugged, it'll know that it's biased towards certain actions or plans over others, and that these situational biases may not be desirable. So it'll know to correct its behavior to suit. (Note that mere awareness of an altered state is insufficient: feedback needs to be detailed enough to allow that course-correction.)
Second, it allows it to predict in detail its future plan-making instances. What, given a plan, its future selves would want to do at various stages of that plan, and how capable they'll be of doing this.
Concretely, self-awareness is what allows to:
To be clear, those are just examples; what we want is the ability to display such meta-planning universally. We can imagine an animal that instinctively shies away from certain drastic actions while angry, but the actual requirement is the ability to do that in off-distribution contexts in a zero-shot regime.
On that note... I'll abstain from strong statements on whether various animals actually have self-models complex enough to be morally relevant. I suspect, however, that almost no-one's planning algorithms are advanced enough to make good use of qualia — and evolution would not grant them senses they can't use. In particular, this capability implies high trust placed by evolution in the planner-part: that sometimes it may know better than the built-in instincts, and should have the ability to plan around them.
But I'm pushing back against this sort of argument. As I've described, a mind in pain does not necessarily experience that pain. The capacity to have qualia of pain corresponds to a specific mental process where the effect of pain on the agent is picked up by a specialized "sensory apparatus" and re-fed as input to the planning module within that agent. This, on a very concrete level, is what having internal experience means. Just track the information flows!
And it's entirely possible for a mind to simply lack that sensory apparatus.
As such, in terms of empirical tests for sentience, the thing to look for isn't whether something looks like it experiences emotions. It's whether, while plan-making, that agent can reason about its own behavior in different emotional states.
1. Cogito, Ergo Sum. It's easy to formalize. As per Dehaene, all of the inferences the brain makes about the outside world are probabilistic. When presented to the planner, they would be appropriately tagged with their probability estimates. The one exception would be information about the brain's own continued functioning: it would be tagged "confidence 1". After all, the only way for the self-awareness mechanism to become compromised involves severe damage to the brain's internals, which is probably fatal. So evolution never had cause to program us to doubt it.
And that's why we go around slipping into solipsism.
2. "Here's a simple program that tracks its own state. Is it sentient?" No. It needs to be an agent.
3. The Hard Problem of Consciousness. None of the above seems to address the real question: why does self-awareness seem so... metaphysically different from the rest of the universe? Or, phrased more tractably: "Why does a mind that implements the self-awareness mechanism start viewing self-aware processes as being qualitatively different, compared to other matter? And gets so confused about it?"
I'm afraid I don't have a complete answer to that, as I'm having some trouble staring at the thing myself. I feel confident, though, that whatever it is, it wouldn't invalidate anything I wrote above.[2] I suspect it's a combination of two things:
And this "qualia" concept has some really confusing properties. It's basically defined by "this thing has first-person experiences, just like me". Yet we can imagine rocks to have qualia, and rocks definitely don't share any of our algorithmic machinery, especially the self-awareness machinery. At the same time, we can imagine entities that share all of our algorithms — p-zombies — which somehow lack qualia.
Why is that so? What is the purpose of this distinct "qualia" concept? Why are we allowed to use it so incoherently?
I'm not sure, but I suspect that it's a short-hand for "has inherent moral relevance". It's not tied to "is self-aware" because evolution wanted us to be able to dehumanize criminals and members of competing tribes: view them as beasts, soulless barbarians. So the concept is decoupled from its definition, which means we can imagine incoherent states where things that have what we define as "qualia" don't have qualia, and vice versa.
It's not a particularly satisfying answer, granted. But what's the alternative here? If this magical-feeling "first-person perspective" can act on the world, it has to be implemented within the world. The self-awareness as I described it seems to suffice to describe all characteristics of having qualia, and the purpose of our capability for self-awareness; it just fails to validate our feeling that qualia are metaphysically significant. But if we can't accept "they're not metaphysically significant, here's how they're implemented and why they falsely feel metaphysically significant", then we've decided to reject any answer except "here's a new metaphysics, turns out there really are etheric parasites making us hallucinate!".
Maybe my specific answer, "qualia-having is a signifier for moral relevance", isn't exactly right; and indeed, it doesn't quite feel right to me. But whatever the real answer is, I expect it to look very similar, and to similarly dismiss the "hard problem" as lacking substance.
4. Moral Implications. If you accept the above, then the whole "qualia" debacle is just a massive red herring caused by the idiosyncrasies of our mental architecture. What does that imply for ethics?
Well, that's simple: we just have to re-connect the free-floating "qualia" concept with the definition of qualia. We value things that have first-person experiences similar to ours. Hence, we have to isolate the algorithms that allow things to have first-person experiences like ours, then assign things like that moral relevance, and dismiss the moral relevance of everything else.
And with there not being some additional "magical fluid" that can confer moral relevance to a bundle of matter, we can rest assured there won't be any shocking twists where puddles turn out to have been important this entire time.