39 Inverting qualia with group theory

1st Dec 2025

9 min read

39

David Chalmers describes the inverted qualia thought experiment, in The Conscious Mind, as an argument against logical supervenience of phenomenal experience on physical states:

one can coherently imagine a physically identical world in which conscious experiences are inverted, or (at the local level) imagine a being physically identical to me but with inverted conscious experiences. One might imagine, for example, that where I have a red experience, my inverted twin has a blue experience, and vice versa. Of course he will call his blue experiences "red," but that is irrelevant. What matters is that the experience he has of the things we both call "red"---blood, fire engines, and so on---is of the same kind as the experience I have of the things we both call "blue," such as the sea and the sky. The rest of his color experiences are systematically inverted with respect to mine, in order that they cohere with the red-blue inversion. Perhaps the best way to imagine this happening with human color experiences is to imagine that two of the axes of our three-dimensional color space are switched---the red-green axis is mapped onto the yellow-blue axis, and vice versa. To achieve such an inversion in the actual world, presumably we would need to rewire neural processes in an appropriate way, but as a logical possibility, it seems entirely coherent that experiences could be inverted while physical structure is duplicated exactly. Nothing in the neurophysiology dictates that one sort of processing should be accompanied by red experiences rather than by yellow experiences.

There are quite a lot of criticisms of this sort of argument. Chalmers addresses some of them, such as the idea that this isn't neurologically plausible for humans (he brings up aliens with more symmetric color neurology as a counter). A principled approach to criticism is to track the semantics of "looking red" through mutually interpretable language, as Wilfrid Sellars does in Empiricism and the Philosophy of Mind. This is, unfortunately, somewhat laborious, and could easily fail to connect with qualia realist intuitions.

I'll take a more philosophically modest approach: analyzing the consequences of the hypothetical, with group theory. Hopefully, this will make clearer requirements that must be satisfied in the hypothetical, and make progress towards isolating cruxes.

Let's start by considering a slightly broader space of color qualia operations that include red/blue inversion. We could think of a color in standard form as a triple of numbers in RGB order. Call an operation that permutes the channels (e.g. swapping red and blue) a channel permutation. The group of channel permutations is , the symmetric group of permutations of a 3-element set. We can write the channel permutations as RGB for the identity, BGR for red/blue inversion, BRG for a red-to-green rotation, and so on. Channel permutations compose as, for example, $G R B \cdot B G R = G B R$ ; group composition "applies the right element first". Each channel permutation can be categorized as either being the identity, swapping two channels, or rotating channels in either the red-to-green or green-to-red directions; there are 6 elements of $S_{3}$ in total.

At a high level, we will construct the color qualia space category CQS as the category of functors $[S_{3}, S e t]$ , where $S_{3}$ is the group construed as a single-object category. This is, of course, highly abstract, so let's go step by step.

A color qualia space has an associated set of elements. Intuitively, these represent data structures that contain colors. The color qualia space also specifies a way to apply $S_{3}$ group elements to these set elements. More formally, a color qualia space is a pair $(Q, p)$ where $p : S_{3} \times Q \to Q$ is group homomorphic in its first argument: $p (R G B, q) = q$ and $p (a \cdot b, q) = p (a, p (b, q))$ . Since $p$ is generally clear from context, we also write $p (a, q)$ as $a * q$ . (Note, p is a group action).

As an example, 100 by 100 images, where each pixel has numbers for each of the three color channels, form a color qualia space, where the group action (permuting channels) maps across each pixel. The function $q \mapsto G R B * q$ performs a red/green swap on its image argument, like a shader.

We want to consider maps between color qualia spaces, but we need to be careful. In the inverted qualia thought experiment, we could imagine that the original and their twin both look at a stop sign, and then yell "RED" if the stop sign's corresponding color qualia are closest to the red primary color. But then the original and the twin would behave differently, contradicting physical identicality. Going with the hypothetical, their mental operations on their color percepts can't disambiguate the channels too much. In some sense, operations mapping qualia to qualia (such as, taking their visual field and imagining transformations of it) have to be working isomorphically despite the red-blue inversion.

The concept of equivariance in group theory is a rather strong version of this. In this case, an equivariant map $f : Q \to R$ between color qualia spaces $Q, R$ has, for any $a \in S_{3}$ , the equality $f (a * x) = a * f (x)$ . Intuitively, this means the function acts symmetrically on channels, not picking out any one as special, and not identifying the chirality of the channels. If the twin's qualia were red/blue inverted from the original's before the equivariant map, they remain so after both apply the map.

Color qualia spaces and equivariant maps between them form a category, $C Q S ≅ [S_{3}, S e t]$ . Let's quickly list some examples of equivariant maps:

Mapping an image to its gray-scale variant.
Setting the right half of an image to gray-scale.
Mapping a function from channel value to channel value (e.g. restricting to a range) over all channel values in an image.
Taking the dominant primary color of the top-left pixel of an image, and then inverting the two other primary colors' channels throughout the whole image

And some examples of non-equivariant maps:

Mapping an input image to an output image representing text spelling "red", "green", or "blue", depending on the dominant primary color in the input image.
Mapping an image $q$ to $b * q$ for a non-trivial $S_{3}$ element $b$ , i.e. swapping two channels, or rotating channels.

Let's examine the last point. Suppose $f (y) = b * y$ is a function on images. Now to check equivariance, we ask if $\forall a \in S_{3}, \forall x, b * a * x = a * b * x$ . But this is only true when $b = R G B$ , the group identity. Note $S_{3}$ is not Abelian.

What are the philosophical consequences? We could vaguely imagine that both the original and the twin mentally rotate red qualia towards green qualia, green qualia towards blue qualia, and blue qualia towards red qualia (perhaps imagining a transformation to their visual field). But this operation (BRG) does not commute with the original inversion (BGR).

Suppose a third person tells both the original and the twin: "Imagine applying BRG to your visual field". The original interprets this "correctly", so actually does BRG. But the twin thinks, "To apply BRG, in my red (color of blood) channel" --- actually blue qualia --- "I put the value of my blue (color of ocean) channel" --- actually red qualia. So the twin actually implements the opposite rotation, GBR!

If you and the twin could both actually apply BRG, and "naively" apply it (in the way that causes the twin to do GBR as above), then you would get the same results naively and actually, while the twin would get different results. Presumably the twin would notice this difference, so we must reject some premise (the original and the twin are supposed to behave the same). And of course, naive application is more straightforward. So we must conclude that you and the twin can't actually both apply BRG.

Let's further characterize equivariant maps. Given a color qualia space $(Q, p)$ , the orbit of an element $q \in Q$ is the set of elements reachable through group actions, ${a * q | a \in S_{3}}$ . Now let the orbit map $π_{Q} : Q \to Q / S_{3}$ map elements to their orbits, effectively quotienting over channel permutations. The orbit map relates to equivariance in the following way: for any equivariant $f : Q \to R$ , there is a unique function on orbits $g : Q / S_{3} \to R / S_{3}$ commuting, $g \circ π_{Q} = π_{R} \circ f$ .

(Why is this true? Note $f$ must map elements of a single Q-orbit to a single R-orbit, which allows defining $g$ commuting. Any alternative choice would fail commutation on some Q-orbit.)

An orbit is itself a color qualia space (a subspace of the original), and must have size 1, 2, 3, or 6 (by sub-group analysis). We can characterize orbits of a given size as isomorphic to a standard qualia space of that (finite) size. Explicitly:

The size-1 qualia space $C Q_{1}$ has elements ${\emptyset}$ ; it is trivial.
The size-2 qualia space $C Q_{2}$ has elements ${L, R}$ representing chirality. Channel reflections (RBG, GRB, BGR) flip chirality, rotations preserve it. (Imagine three balls corresponding to primary colors, with sticks connecting them in a triangle; chirality-reversing operations require flipping the triangle over vertically.)
The size-3 qualia space $C Q_{3}$ has elements {R, B, G} representing primary colors. Group operations work straightforwardly, e.g. $B G R * B = R$ .
The size-6 qualia space $C Q_{6}$ has as elements total orderings of primary colors (e.g. R > B > G), of which there are 6. Group operations work straightforwardly, e.g. $B G R * (R > B > G) = (B > R > G)$ .

Now let's consider equivariant maps between these standard qualia spaces. Any equivariant map between these must be either: some space to itself, some space to $C Q_{1}$ , or $C Q_{6}$ to anything. (For example, there are no equivariant maps from $C Q_{1}$ to $C Q_{6}$ .) So we can reduce the 4 x 4 of signatures $C Q_{i} \to C Q_{j}$ to only 9 realizable signatures. 4 of these signatures are clearly trivial, as they map to $C Q_{1}$ . Exhaustively analyzing the possible equivariant maps of the non-trivial signatures:

There are two equivariant maps $C Q_{2} \to C Q_{2}$ : identity and chirality-reversal.
There is only one equivariant map $C Q_{3} \to C Q_{3}$ , the identity.
There are two equivariant maps $C Q_{6} \to C Q_{2}$ , which assign distinct chiralities to the two cyclic orbits, {(R > G > B), (B > R > G), (G > B > R)} and {(B > G > R), (R > B > G), (G > R > B)}.
There are three equivariant maps $C Q_{6} \to C Q_{3}$ , which pick out either the greatest, middle, or least primary color.
There are six equivariant maps $C Q_{6} \to C Q_{6}$ corresponding to each of the $S_{3}$ permutations applying to the ordering, e.g. swapping first and second places.

In tabular form, cardinalities of equivariant map sets $| H o m_{C Q S} (C Q_{i}, C Q_{j}) |$ are as follows:

in \ out	$C Q_{1}$	$C Q_{2}$	$C Q_{3}$	$C Q_{6}$
$C Q_{1}$	1	0	0	0
$C Q_{2}$	1	2	0	0
$C Q_{3}$	1	0	1	0
$C Q_{6}$	1	2	3	6

We now have a combinatorial characterization of equivariant maps in general. First determine how $f$ maps input orbits to output orbits. Then for each input orbit $π_{Q} (x)$ , determine how $f$ equivariantly maps it to the corresponding output orbit $π_{R} (f (x))$ , which has a finite combinatorial characterization.

In the reverse direction, we can form a valid equivariant map by first choosing a map on orbits $g : Q / S_{3} \to R / S_{3}$ , then for each input orbit, selecting an equivariant map to the output orbit. Formally, we could write the set of such maps as:

$H o m_{C Q S} (Q, R) ≅ Π_{S \in Q / S_{3}} Σ_{T \in R / S_{3}} H o m_{C Q S} (S, T)$

where $H o m_{C Q S} (S, T)$ is the set of equivariant maps between color qualia spaces S and T, and $Π, Σ$ are set-theoretic dependent product and sum.

We now have a fairly direct characterization of equivariant maps in CQS. They aren't exactly characterizable as "functions between quotient spaces" as one might have expected. Instead, they carry extra orbit-to-orbit information, although this information is combinatorially simple for any given pair of orbits.

What are the philosophical implications? To the color qualia realist, CQS decently characterizes mental operations on color qualia that don't break the physical symmetry, in thought experiments such as inverted qualia. Equivariance is mathematically natural, and rules out non-realizable mental operations such as BRG. Analysis of CQS, including combinatorial characterization of equivariant maps, provides a functional analysis relevant to the physics of the situation, which (according to Chalmers) doesn't actually involve color qualia as physical entities.

To the color qualia non-realist, the functional characterization of color qualia intuitions through CQS could provide a hint as to a specific error or illusion the color qualia realist is subject to. The non-realist could expect that, once the functional physical component to the intuition is characterized, there is not a remaining reason to expect color qualia to exist above and beyond such physics and physical functions.

I have found CQS to be clarifying with respect to the inverted qualia thought experiment: equivariance provides a mathematically simple constraint on realizability of mental operations in the scenario. In particular, CQS analysis led me to correct an intuition that channel rotations such as BRG would be realizable, and the combinatorial characterization showed (non-obviously) that maps between quotient spaces are insufficient. My own view is that inverted qualia arguments are fairly weak, and that CQS analysis has some relevance to showing the weakness, but the fuller case would require engaging with the relationship between phenomenal experience and belief-formation.

(If you like the idea of a circular "color wheel" rather than a three-channel "color cube", you may consider the (also non-Abelian) orthogonal group O(2), and the continuous qualia space category $[O (2), S e t]$ .)

World Modeling

Frontpage

39

New Comment

23 comments, sorted by

top scoring

Click to highlight new comments since: Today at 8:44 AM

[-]James Diacoumis2mo40

This is a really interesting post.

My own view is that inverted qualia arguments are fairly weak, and that CQS analysis has some relevance to showing the weakness

Can I clarify why you think this? My main takeaway from the post is that naive colour swaps aren’t possible while keeping the structure fixed, but there are more sophisticated equivariant operations on the toy spaces that do preserve the structure.

In other words, while the initial intuition behind inverted qualia is a bit loose — the core idea that some change to the qualia while leaving structure fixed is logically coherent and actually supports the phenomenal realist claim.

[-]jessicata2mo40

To be clear one can do something like "imagine swapping red and blue" equivariantly, you just have to be looking at, or recall in memory, control red and blue (physical) objects. (This is like the "naive BRG rotation" mentioned)

On the more general point, there seems to be some reason why qualia theorist really like color. Inverted qualia, "redness of red", and Mary's room. My suspicion is that there is something specifically confusing about color, which makes qualia realism seem relatively strong in this domain, while weaker in other domains. This probably has to do with CQS-like group theory.

I'm not convinced CQS provides much new evidence, but I thought previous evidence indicates that the color qualia non-realist wouldn't have too much trouble with color; Chalmers types would already agree there is a physical, functional explanation for the physical situation that doesn't involve color qualia. What CQS and similar can do is make the general area of color less confusing.

This could lead to a clearer picture of both "what color qualia are, if they exist" and "what produces the illusion of extra-physical color qualia". My guess is that the second, modestly illusionist picture is going to look more compelling, though that's hard to establish ahead of time.

Someone linked me to Yuxi's Hole Argument and Inverted Qualia, which I hadn't read before writing this post; it is more philosophically ambitious while having a broadly compatible group-theoretic picture.

[-]James Diacoumis2mo40

To be clear one can do something like "imagine swapping red and blue" equivariantly, you just have to be looking at, or recall in memory, control red and blue (physical) objects. (This is like the "naive BRG rotation" mentioned)

Thanks for the clarification here.

On the general point, I acknowledge that you're being pretty philosophically modest with this framework. I'd just flag that my intuition actually goes in the opposite direction and I find it pretty strongly suggestive of phenomenal realism.

If I understand CQS (and Yuxi's post that you linked) you're formalising a structure which encodes the tranformations that leave physical observables unaffected. In CQS, these are equivariant mappings on the qualia space, in Yuxi's post these are diffeomorphisms on the spacetime manifold. There's another nice example of electromagnetic fields being invariant under U(1) gauge symmetry which fits the same pattern.

My intuition is that these all work to describe physical systems precisely because they're good third-person descriptions of the world. Once you give a first-person description of the world (consciousness) this is analogous to selecting a coordinate-system or fixing the gauge potential. You pick out one element of the equivalence class as "what it's like for me now."

To link it up with your post, my intuition is that the way red (colour of blood) or blue (colour of water) look to me as a first person observer are fixed and not fully specified by the structure. They need to be "populated" with something. Very roughly, I'd guess the crux is that the illusionist would say this "population" is entirely done by the brains' functional/relational states whereas the realist would say there's some categorical property in the world which is populating this structure.

[-]jessicata2mo*40

I would agree that, at least shallowly, CQS provides weak evidence for qualia realism, in that it failed to find a severe theoretical problem, while an illusionist may have expected that.

In the case of special relativity, one might naively expect that "objects objectively have lengths". However their lengths differ depending on relative velocity of the reference frame. So the naive intuition was invalid. You can still ask "how long is that object from my reference frame?", indexically. Though probably there's a better, deeper description of reality than in terms of centered world models corresponding with the reference frames (acted on by a symmetry group).

The case of qualia inversion seems somewhat worse than this. As long as we could imagine the twin having inverted qualia, why not imagine them inhabited by multiple consciousnesses, whose qualia span the entire orbit (of size 6)? Then there isn't a fact of the matter of 'what is the twin's qualia space element', the twin (physical) corresponds to 6 different conscious perceivers in an orbit. The perceivers would, if correct, have to realize there is not a fact of the matter of 'which orbit element is the consciousness corresponding to this body perceiving'. (This is a bit more of an 'inflationary' direction of hypothetical, compared to 'deflationary' standard physicalism.)

(Note, to get to behavior you are basically equivariantly mapping to a "Shannon" qualia space, by which I mean a space where the group action does nothing, all orbits are singletons. Equivariantly mapping to a Shannon qualia space preserves only information about which orbit. Once it's in a Shannon space it's valid to convert it to specific behaviors and so on, without worrying about equivariance. So at the level of behavior, it's only necessary to track 'which orbit'.)

I think there is some reason to expect that either (a) qualia inversion isn't really possible or (b) there's a deeper theory than positing different color-permuted reference frames. In the relativity case it seems like initially differentiating/expanding the reference frames did real work, even if there's eventually a more unified theory. But in the color qualia case it seems less like it's doing work; it doesn't seem to yield predictions for the physicalist, while relativity sure does yield new predictions for the Newtonian.

[-]James Diacoumis2mo20

The comment about multiple consciousness spanning the entire orbit in a single physical system is a very cool idea. I’m broadly sympathetic to this kind of approach in other areas e.g. the Many Worlds Interpretation where all branches are real but only one is indexed.

To clarify my “population” comment above - I think a phenomenal realist could still push back here and ask what makes something intrinsically R rather than say R’ in the first place. Why not say there are R’, G’ and B’ which are distinct experiential states all compatible with the same structural/relational profile? On that picture R has an intrinsic property which distinguishes it from R’ but is not fully specified by the structural properties.

I share your intuition that deep physical theories usually exhibit a high degree of symmetry and that we’re often right to discard naive intuitions about “fixing the base” in order to build better third-person descriptions of the physical world. But I also think there’s a strong urge to pattern match based on the past success of physical theories which are inherently third-person descriptions. Since consciousness is a first-person phenomenon I think there’s at least a principled reason to think there might be some “intrinsic” properties which are missing from the purely structural and relational properties that physics gives.

[-]jessicata2mo40

So I think the physicalist-leaning person has an answer as to "what it means to have an R experience", e.g. "to have the sort of internal representation people tend to have when viewing standard red objects (e.g. stop signs, ~700nm light) under standard conditions" (this is a Sellars-inspired formulation, though he'd have something more precise; I realize my definition probably works badly in some edge cases). Then the inverted qualia thought experiment is rejected on semantic grounds. (In particular there appears to be no way to semantically refer to the difference between R and R' through inter-subjective language)

I realize this might seem like begging the question to the qualia realist; it is more demonstrating that it's possible to adequately use qualia non-realist language, not that qualia aren't real. It does open the question of what the qualia realist is imagining when thinking that R and R' are distinct experiential states.

The CQS picture could say: you can equivariantly imagine swapping red and blue in your visual field (i.e. experiences had when seeing standard red/blue objects), then think you're imagining the experience of a possible physically identical twin. The thing is, if you let stand for that equivariant color swap map and $x$ for your present color qualia, then you're not noticing the difference by looking at $f (x)$ alone, you're more mapping $x$ to $(x, f (x))$ and then noticing an internal distinction on the output pair. That's what allows you to produce Shannon information "I imagine f(x) != x".

The difference is effective in this case; when you put the imaginations of $x$ and $f (x)$ side by side, you can effectively (through an equivariant map to a Shannon space) find the difference. The thing is, the difference between the original's and the twin's experience can't be effective in any way. If they could "brain meld" and share neurological bases of color qualia (visual cortex info or whatever), they could at no point see that one has $x$ and the other has $f (x)$ , and produce Shannon information indicating the distinction.

This seems to indicate some confusion where the initial intuition that they would have different experiences was based on imagining an omniscient perspective "original has $x$ , twin has $f (x)$ ", an omniscient perspective which can (equivariantly) notice the difference. But there's no way to instantiate a perspective that notices the difference, by mad science brain-melding and so on. So maybe the initially imagined distinction was illusory. It's hard for me to say exactly why this indicates a problem, but it seems problematic nonetheless.

[-]James Diacoumis2mo40

I agree that this is a coherent illusionist picture and I also agree that the phenomenal realist will find it question begging so maybe this is where we can isolate the crux?

I agree that if we define “red experience” using a Sellars-style definition in terms of the functional role that red plays in our language and behaviour then the R vs R’ distinction gets blocked. There’s no functional difference between the two so on that semantics they refer to the same thing.

By contrast, the phenomenal realist would define “red experience” using a Kripke-style rigid designator for whatever intrinsic property actually underlies the red region of quality space in my world.

To put it another way, imagine two counterfactual worlds

World A - the full physics is described with the same CQS, same orbits etc… and the base property which instantiates red regions of quality space is R.

World B - exact same physics, same CQS, same orbits as world A but the base property instantiating red in the quality space is R’ which is intrinsically different to R.

Inside each world the phenomenal realist would be happy to grant that there’s no way to differentiate R from R’. They’d also agree there’s no way to instantiate a perspective that would notice the difference, via omniscience or brain melding or whatever.

Nonetheless, they’ll still feel justified saying there’s an intrinsic difference between R and R’ in terms of the rigidly designated experience that is actually instantiated in their world.

I fully take your point that this could indicate that the imagined distinction is illusory. I’d actually count this exchange as a small update towards illusionism on my part because the urge to pattern match on past successes of physics is so strong, but I’m still inclined to press the objection that physics only fixes the third-person relational structure of the world but leaves the first person “intrinsic” nature of what instantiates the structure untouched.

[-]jessicata2mo40

I think the illusionist move here is to, while doubting that the base property distinguishing world A from world B really exists, go with the hypothetical to find a semantic issue. You're saying there is a way to "rigidly designate experience", referencing the base property, through Kripke style semantics. I doubt this, even on a view where such base properties exist.

From having read Naming & Necessity (though not super carefully), I think a paradigmatic case of Kripke semantics is referring to a length such as a meter. What do I mean by a meter? "One meter" rigidly designates the length of the standard meter in Paris. Once you look at how I said "one meter" and look at the physical world, you can figure out what length I mean. That is a step from a string "one meter" to an actual length.

Another example: Twin Earth thought experiment. Oscar is on Earth where the watery substance is H2O. Twin Oscar is on an alien planet where the watery substance is XYZ. Both say "water" and rigidly designate different chemicals. Even if Oscar and Twin Oscar have the same functional mind states, they mean different things by way of rigid designation.

Both these cases of rigid designation work, because you can look to the physical world to find the "realistic property" corresponding to the string ("one meter" or "water"). With color qualia, I simply don't see any way to do that, even buying into the idea that there could be a base property that differs between worlds A and B. As an example, with inverted qualia, it doesn't seem like a third party can rigidly designate a color qualia by pointing to anything in the environment (analogous to the standard meter, or watery substance); any object they point to would, under these stipulations, look different (have different color qualia) for the original and twin, so it wouldn't actually pick out the base property.

[-]James Diacoumis2mo20

I’m following Chalmers’ 2D semantics here and specifically pushing the Russellian Monism line.

To unpack what I mean, imagine that the intrinsic property R stands to “redness” in roughly the same way that H20 stands to water for Oscar and R’ stands to some other property (call it “shredness”) in roughly the same way XYZ stands to water for Twin Oscar. Oscar and Twin Oscar would have different experiences of redness vs shredness but you’d never be able to tell this from the outside third-person perspective. Nevertheless, the phenomenal realist would assert that their experiences are still intrinsically different.

Water has a primary intension of “the watery stuff in the lakes and rivers” and a secondary intension of “H20 molecules”. So we can say the sentence ‘water is not H20’ is primarily conceivable but not secondarily conceivable.

The Russellian monist wants to say the same thing about phenomenal (or protophenomenal) properties. The sentence ‘red is not R’ is primarily but not secondarily conceivable. This is what I meant by saying “redness” is rigidly designates the property R.

I think you’re right that the phenomenal realist needs to bite a bullet on how exactly they get epistemic access to R because there’s no way to just look in the world and determine what it is. Most realists would say they’re directly acquainted with it which can sound question begging to an illusionist.

I’m tempted to say that the brain is able to “notice” when it’s tokening a qualitative state R if it develops enough cognitive sophistication. This view is called panqualityism and I find it a pretty attractive view, although it has its own problems (how exactly is “noticing” supposed to work?)

[-]jessicata2mo30

Re Chalmers article:

I wrote Generalizing zombie arguments on the general conceivably/possibility structure, it's familiar.

To be clear, inverted qualia thought experiment fails if either (a) it fails to express semantically some well-formed possible situation (e.g. situation represented as a proposition), (b) it semantically expresses something that is not logically possible (which seems reasonably similar to "ideally conceivable"?)

Rigid designation seems to be the movement from primary to secondary intension, in Chalmers' language. It's a long article so I'm not sure what relevant it adds on top of Kripkean rigid designation.

I accept the primary-to-secondary movement in the twin earth case, but not the inverted qualia case. I think there is an actual disanalogy. There are third party verifiable facts that allow the primary-to-secondary passage to be shown to work differently on Earth and twin Earth. Not so with inverted qualia. So I don't think you can just say they are analogous and specify the semantics that way.

And equivariance of course puts a hard limit on "referring to R"; if either the original or the copy (in inverted qualia thought experiment) could follow the instructions "imagine R", then they could break symmetry. Hence R is not imaginable in any specific way. Which lends credence to the idea that R isn't semantically well-formed in propositions like "My experience is R when I see a stop sign". (Possibly, it is semantically possible to refer to a supposed distinction between R and R', which qualia realists are asserting exists and illusionists are asserting does not exist, such that they are succeeding in really disagreeing; but it does not follow that either R or R' is individually something that can be referred to.)

There is no analogous problem in twin Earth; Oscar can refer to "water", third parties can figure out the rigid designation by looking at the environment, etc. And Oscar can think "H2O" and "XYZ" as hypotheses (imagining molecular structure and so on), there's no "equivariance so you can't actually imagine either possibility specifically" constraint.

I’m tempted to say that the brain is able to “notice” when it’s tokening a qualitative state R if it develops enough cognitive sophistication.

Anything the brain does has to work the same across the orbit, though. You could imagine all 6 observers sitting above the same physical body. It seems a semantic problem to say the brain is tokening a qualitative state R at all. (In original inverted qualia, the original and the twin both have the same brain states, so similarly, speaking of the brain tokening R is invalid; whatever the brain does it must do the same in both cases, and anything about R has to work differently in both cases by stipulation.)

Going to paste in a short Twitter thing I wrote that has some relevance:

The mirrored universe thought experiment

Suppose hypothetically, the universe is deterministic, and has exact mirror symmetry about some plane. Everything on one side has a mirrored counterpart on the other side, evolving identically (but reflected). If you approach the plane, you can see the other half, as if it's through a perfect mirror. (Perhaps photons bounce exactly on mirror-photons, or perhaps physics allows them to pass through each other.)

Now consider, Ray goes towards the plane of symmetry and raises his right hand, saying, "This is my right hand". His mirror copy looks, to Ray, like he raises his left hand, and says, "This is my right hand".

First question: Is Ray correct when he says "This is my right hand", and similarly for mirror-Ray? A symmetry argument suggests that they're both correct, or neither is. But it appears, to Ray and mirror-Ray, that only one is correct.

Second question: What is the experience of mirror-Ray like? Does mirror-Ray have chirality-reversed experience (visual, auditory, tactile)? And if mirror-Ray's experience is chirality reversed, why does he act the same (mirrored) as Ray? Why can't they tell the difference and have different physical behavior?

Suppose Ray gets confused about left/right. He remembers, like most people, he's right handed. So he raises his dominant hand, assured it's his right hand.

Same goes for mirror-Ray. But when mirror-Ray does this, it really looks to Ray like mirror-Ray is raising his left hand.

On a functionalist account, Ray and mirror-Ray have the same experiences. They're not mirrored versions of each other; they are the same experience, as "right" and "left" get their meaning through structural relations, to hand dominance, heart positioning, and so on, not through an absolute spatial quality.

The inverted qualia thought experiment (of Chalmers and so on) asks, what if there were an exact physical copy of someone, who had their color channels permuted, e.g. swapping their red with their blue experience? A similar chiral question: what about an exact physical copy who had their left/right experience mirrored? (And if such a person existed, what would be the relationship between their mirrored experience and mirrored physics?)

[-]James Diacoumis2mo30

I think we've hit a legitimate crux here because the Russellian monist is pushing something that you don't accept. You're insisting that there's a semantic problem because there's no third-person publically observable way to reconstruct what we mean by 'R' in the same way that we have for 'H20'

The Russellian monist is saying that phenomenal concepts are not fixed by that kind of environmental or functional procedure, instead the concept "red" is fixed by acquaintance i.e. by being directly presented with that quality in experience. So no third-party could construct what R is and there's no physically specifiable, equivariant instruction that uniquely produces R rather than R'.

The illusionist will reject this formulation, but the question the phenomenal realist will pose is... why exactly? The Russellian monist thinks that if you insist every referent must be tied to third-person publically observable facts then you lose the only plausible route to picking out first person data points e.g. "the way red feels for me right now".

Your appeal to the 6 observers spanning the orbit is nice because it explains why one of the channels R, G or B feels like it's being singled out. But the Russellian monist will say the structure doesn't fully specify the realisers of the structure. The realisers could equally be (R, G, B) or (R', G', B') or (X, Y, Z) and the structure doesn't tell you anything about the character of what's realising the structure. Your move to reject the semantic difference between R and R' would have us accept that there's no intrinsic difference between the phenomenal character of R and R' so long as they're playing the same functional role in realising the structure. The Russellian monist rejects this.

Consider by way of analogy the game of chess where all the pieces -- rooks, bishops etc.. are defined in terms of their structural/functional role e.g. the rook moves in straight lines, the bishop moves diagonally etc.. and when all the pieces are defined in this way they collectively make up the web of structure called "chess". The analogy here is to say the little wooden pieces which realise the rook and bishop are the intrinsic properties that instantiate the structure. You could imagine changing something about the wooden pieces shape or design but leaving the structure of the game of chess untouched. To be clear, this is an analogy, 'rook' is obviously defined in terms of the functional role that a rook plays in the game of chess and not by the little wooden piece that denotes the rook. The Russellian Monist claim is that a phenomenal concept "red" latches onto the intrinsic realiser of the role, not the role itself.

I understand the illusionist will want to reject this picture, but again, the question is why? From the Russellian perspective, this picture does a coherent job of treating the intrinsic first-person data points of experience.

[-]jessicata2mo30

The chess analogy suggests a comparison to the simulation hypothesis. Perhaps our universe is simulated within another universe. That universe has extra properties that get "attached" to physical coordinates and so on, perhaps implementation details, or perhaps extraneous but "nearby" entities. That raises the question, can we in any meaningful way refer to them?

One thing that suggests "yes" is that perhaps glitches are theoretically possible, which would give away the simulation, and noumenal information leaks into our universe. Or maybe the glitches are just counterfactually possible. Either way, it seems like it's vaguely possible to directionally point at properties of the simulation through counterfactual glitches in the Matrix. Perhaps there's a buffer overflow that prints the source code and program state into our universe, or whatever.

While it seems there's a legitimate obstacle to reference (through the simulation being actually a good simulation), it's not clearly impossible. It seems like, yes, our universe could meaningfully be simulated, and then there would be corresponding specifics.

Here's where the analogy breaks down. Someone could say, "theoretically, I can hypothesize our universe is simulated and uses 256-bit floats for physical spatial quantities". That seems ok as a hypothesis. But it would be a bit strange for them to be asserting that they're directly acquainted with the 256-bit float nature of the simulation's implementation. They can't really distinguish it from 512-bit floats without ridiculously impractical experiments (particle colliders won't do). So it's not really analogous to qualia. The idea of qualia as physics implementation detail breaks down.

So it seems there is a 'rigid designator' story that lets "the universe runs on 256-bit floats" be theoretically meaningful, even if glitches that would allow determining that never actually occur. But this isn't the sort of direct acquantiance that the qualia realist wants there to be.

Here's a different analogy to chess. What about role-playing games? People play, say, a MMORPG with each other. They can claim to be "directly acquantied" with things outside the world. And they can make their characters say things like "I'm going to take a break to go to the park, afk for a while". It doesn't make much sense in-universe, but it's sensible communication anyhow.

Physics is more like if the MMORPG got so detailed that they way you play is that you construct an entire deterministic brain emulation as your avatar. Then the avatar works on its own, deterministically. Now it seems much harder for the players to make the avatars say something like "I'm going to take a break, afk", meaningfully. Any behavior like that had to result from the initial conditions and the update rules. (And a "non-deterministic physical system" that allowed players to intervene would be like interaction dualism.)

It seems like the avatars saying things like "umm, yes, of course I am directly acquainted with 'my laptop model' and 'afk at the park'" can only work communicatively if the semantic info was baked into the initial conditions of the game world. But now we're back at "extremely weird models of free will" (see Aaronson on freebits). The players only have volition through creating the initial setup. If the universe's initial conditions have too low K-complexity, they have no volition, and there is no way for the info to leak in. (Like, if the players get a lot of bits to specify the avatars as brain emulations and so on, I guess semantics is possible, but it doesn't as well map to our universe and standard physics.)

I'm not really sure I've found a great argument against the view you're presenting, but it's more like, while it has some initial intuitive plausibility, when you fill out the details of how the semantics work, the picture starts to look extremely strange, and ability to make semantic references is at least pretty dubious.

[-]James Diacoumis2mo10

I'd resist the idea that being embedded in a separate universe as in the simulation hypothesis is necessary for any of this to go through. That feels like it’s smuggling a kind of God’s-eye perspective back in: you’re trying to tell the difference between R and R′ by looking for some third-person, relationally detectable difference. But by hypothesis there isn’t any such third-person difference. You only get a grip on R vs R′ by tokening the states from the first-person perspective and being acquainted with them.

Even though I don’t think the simulation setup is necessary for the argument, I agree it’s a useful way of getting a handle on the dialectic. I’m totally happy with your framing where sim-words refer to things inside the simulation rather than in base reality. When a sim says the word ‘park’ they’re really referring to a sim-park^[1].

You're right that if we were relying on the intrinsic realiser outside the simulation to ground the phenomenal properties that we'd indeed have a semantic problem, but this isn't the move. The Russellian identifies the physical properties within universe such as whatever plays the mass-role and charge-role and says these have an additional intrinsic property which is not fixed by the causal/functional role that they play. If mass was instead realised by something different (say pseudo-mass or schmass) then it would really be different in a meaningful sense even if it caused no third-person change to the physics.

Crucially for your point, if we're in a simulation then we're talking about sim-mass and sim-charge rather than mass and charge in the base reality. So I don't think we run into trouble with semantics and I don't think we need any "information leakage" from the host system to make sense of the terms.

On your specific point:

But it would be a bit strange for them to be asserting that they're directly acquainted with the 256-bit float nature of the simulation's implementation.

I agree this would be weird and it's not what the Russellian is going for. They're saying they're directly acquainted with a quality in experience i.e. "this red" and the phenomenal character of this experience fixes its primary intension. This quality also has a secondary intension which rigidly designates the categorical base property R. So they don't have direct access to R, in the same way Oscar doesn't have direct access to H20 molecules when he's looking at the watery stuff in the rivers and lakes.

The point of this Russellian picture is to show that there’s metaphysical room to explain the phenomenal character of experience in terms of categorical, intrinsic properties in the physical base without violating the causal closure of physics. We’re not appealing to spooky interactionist dualism or contingent psychophysical laws tacked on over and above the physical. The explanatory work is done by the intrinsic nature of the physical properties themselves.

And I think this poses a challenge for the illusionist. Why not accept this picture? It gives a coherent story that takes the first-person data points at face value, rather than explaining them away. I can understand holding onto illusionism if there were no metaphysical room for such phenomenal properties, but the Russellian picture shows how such room can exist without violating causal closure.

Finally, I have a bit of a challenge for you. How exactly do you specify the "base" that's instantiating the structure on your view? If structure is only defined relationally, then what are the relata? It seems you have three options:

Accept that there are no relata are at all. It's relations all the way down.
Accept that there are relata, but they're not phenomenal.
Accept that there are relata, and they're phenomenal.

The Russellian view is 3). The question if you accept 2) is why? We have a coherent metaphysical package that can accommodate the first-person data points, so what motivates insisting that the base is non-phenomenal if it's not doing any work?

So the alternative I see you pushed towards is 1) i.e. biting the bullet that there are no relata at all and that only abstract relational structure exists. Are you happy with that view? To be sure, it's a coherent view in the literature called Ontic Structural Realism but I find it a very hard bullet to bite.

^{^}
As a fun sidenote, I think there are some words like ‘communication’ or ‘computer’ that have the same meaning in sim-world and base reality, but I don’t think that detail matters much for this discussion.

[-]jessicata1mo*20

The Russellian identifies the physical properties within universe such as whatever plays the mass-role and charge-role and says these have an additional intrinsic property which is not fixed by the causal/functional role that they play. If mass was instead realised by something different (say pseudo-mass or schmass) then it would really be different in a meaningful sense even if it caused no third-person change to the physics.

This seems reasonably analogous to the 256-bit float example. That's what I'm saying is dis-analogous from qualia/experience.

They're saying they're directly acquainted with a quality in experience i.e. "this red" and the phenomenal character of this experience fixes its primary intension. This quality also has a secondary intension which rigidly designates the categorical base property R. So they don't have direct access to R, in the same way Oscar doesn't have direct access to H20 molecules when he's looking at the watery stuff in the rivers and lakes.

It's hard for me to read this as compatible with the qualia realist view. The phenomenal character of red is the primary intension, so far so good. But the secondary intension designates something distinct from the phenomenal character, which is called R. It's an implementation detail (like 256-bit floats?) that the person doesn't know about. It's ok to say they can reference that, as they can reference 256-bit floats, or H20 and so on, with a primary intension not containing the details.

The problem is that the qualia realist wants to say that R is the phenomenal character of red. The original and the twin in the inverted qualia thought experiment are supposed to have different phenomenal experiences. If they had the same phenomenal experience but different extra-physical implementation details (note, this is not inherently contradictory bc physics could be scoped to quotient over some of these), that is not satisfying the requirements of the thought experiment. The issue is that R is, in this view, not the phenomenal character of red, but an implementation detail.

(Analogously, with twin Earth, Oscar and twin Oscar could have the same phenomenal experience, and the same primary intension, despite different secondary intensions. That suggests phenomenal experiences go with primary intensions, not secondary intensions. They of course can't do the science required to distinguish the secondary intensions by looking at their experience introspectively.)

Finally, I have a bit of a challenge for you. How exactly do you specify the "base" that's instantiating the structure on your view?

So we have to distinguish between (a) my actual view (b) the view I'm stipulating, generously to the qualia realist picture, to show a semantic issue

Regarding (a) I think something like, "physics picks out a homomorphic structure in reality" and "mind picks out a homomorphic structure in reality", where it's not clear what the right category is, but it should have computational properties not just model-theoretic properties. (See post on homomorphic encryption) Now even though I'm validating neutral monism to get computational properties, I still don't think this validates color qualia in the sense of Chalmers.

Regarding (b) what I'm stipulating is that the relevant category theoretic structure is something like CQS. The base properties "exist" in a sense weaker than how Shannon information exists. Whenever you convert to Shannon information (equivariantly), you only preserve "which orbit you are in". And this is a bit more speculative, but there's going to be way to study CQS that don't reify the base properties, as an alternative formulation. I think on this view you still have trouble "referring to R", on relatively standard semantic views like "you refer to things by saying information specifying them", requiring a significant weakening of semantics to get the references to work out.

Now when you present the alternatives with 'relata' I actually don't know what you mean. At a high level I am taking a category theoretic view where there are type-like objects and function-like morphisms. But the objects don't exactly have to be sets, or contain sets; there can be equivalent descriptions. And reifying everything as "existing" is perhaps a bit much metaphysically.

A basic realist picture is that propositions in general have truth values (as long as they are well formed). So a minimal realist view is that reality maps propositions to truth values (being representable as elements of Cantor space). But then you notice that there are different isomorphic languages for the propositions. And so you get more of a category-theoretic view that way.

I'm dubious about the existence of "multiple substances" in the classical philosophical sense. There is a "syndiffeonesis" argument that for things to be different, they have to have something in common. And as long as they have something in common, what is the meaning of claiming they have "multiple substances"?

Now I'm not sure if "multiple substances" is like "relata". But in any case, my view contains at least Shannon information and resource-bounded Turing computation. (Which is not to claim that they are fundamental, they could be homomorphic from a fundamental structure)

And also I'm in general skeptical of the idea of things "directly presented to us" on Sellars-type grounds (myth of the given). The case of "directly acquainted with the redness of red, which could be switched with the greenness of green while keeping the rest of physics the same" is an especially dubious case. This is the sort of consideration that makes me doubt "relata being phenomenal" in general, even grating relata.

Reading the SEP article, ESR seems about true, since we couldn't know if we're looking at a model of a given theory, or an equivalent model of an isomorphic theory (this is the category-theoretic reading). And more generally we could suppose, what we think is reality or physics could be something which the real reality has a homomorphism to. (Simulation hypothesis is a crude example; virtual embedding as homomorphism)

OSR seems like a somewhat dubious meatphysical hypothesis, although perhaps a methodologically useful one. That is, finding explanations that have only relations no individuals might be useful, but would require something like category-theoretic tools to find isomorphic structures that don't reify sets (at objects and so on). Arguably, if we have the isomorphic structure then we can Occam's razor our way into "no relata being related!" but the devil is in the details.

Rather, my view is that different views (inculding ones that have relata) can be developed, one can find isomorphisms/homomorphisms between them, and one should doubt finding "the Real True view" (rather than, some homomorphism from the Real True view to a true-ish real-ish one) for standard Humean/Kantian reasons.

[-]James Diacoumis1mo30

When I said “they don’t have direct access to R” this was imprecise and invited reading R as an implementation detail. It’s not an implementation detail so I should clarify precisely what I mean here.

The phenomenal character of red in my experience and the categorical base property R are the same thing. So when I have a red experience, in a sense, I’m directly acquainted with R as this quality in experience (which is the primary intension.) What I meant to deny is not direct acquaintance with R but rather a transparently rendered a priori access to R under a physical/structural description that would let you derive what the phenomenal character is like. In other words, you need to be literally tokening the property R from a first-person perspective to experience the phenomenal character of red. The secondary intension just rigidly designates the property R across all possible worlds.

The 256-bit float vs 128-bit floats example is disanalogous because there’s a structural implementation difference in the host system which is causing the change. R and R’ have no internal structure with which to differ, instead they differ intrinsically. Think of it like the mass-role in physics. If we switched the intrinsic property of the mass m with m’ such that F = ma now read F = m’a the physicist would say that nothing has changed. Whether m or m’ is playing the mass-role leaves the third-person physical observables untouched. The Russellian move is to say that there's still a further fact about the categorical properties m and m' that cause them to differ intrinsically.

I think on this view you still have trouble "referring to R", on relatively standard semantic views like "you refer to things by saying information specifying them", requiring a significant weakening of semantics to get the references to work out.

Your semantics is doing a lot of work here, and I wouldn't grant that it's standard. If you build semantics in a way that reference always goes via informational differences in the structure, then of course reference to intrinsic properties will look impossible. From my perspective, this is baking the illusionist/structuralist conclusion into the semantics and I'd treat that as a bug rather than a feature.

The motivation for this Kripke-Chalmers style 2D semantics is motivated by these kinds of cases where we do seem to latch onto things whose immediate microstructure we don't know (e.g. water/H20). And I'd argue that this acquaintance-based form of semantics works for all sorts of things like "this pain", "that red" and arguably even "that object" without needing informational differences to specify them up to some structural isomorphism.

Regarding the term relata, I'm just using it to mean "things" which stand in relation to each other. On view 1) ontic structural realism says there are no "things" which stand in relation to each other, it's the relations themselves which exist and that's all. Your view sounds to me like it leans towards 1) with some epistemic humility about whether a richer structure like 2) really underlies reality.

I share your intuition that OSR sounds like a metaphysically dubious hypothesis even if it's methodologically useful and given that your semantics is strongly information-based, it's unsurprising that talk of "things" or "relata" feels slippery because your semantic machinery doesn't have the resources to pick them out. That's exactly why I'm inclined to bring acquaintance-based reference to intrinsic categorical properties as an extra ingredient.

I'm dubious about the existence of "multiple substances" in the classical philosophical sense. There is a "syndiffeonesis" argument that for things to be different, they have to have something in common. And as long as they have something in common, what is the meaning of claiming they have "multiple substances"?

I’m happy to grant that talking about “phenomenal substance” vs “physical substance” in the Cartesian sense is not well-formed. What matters more for me is just the distinction between intrinsic properties and relational/structural properties. Once we’ve granted that reality is not purely structural and there are some categorical/intrinsic aspects to reality then the distinction between 2) and 3) starts to collapse: they both allow relata with an intrinsic nature, 3) just treats our first-person acquaintance with experience as evidence about what that intrinsic nature is like.

So I think there are two cruxes:

Do we allow intrinsic/categorical properties in addition to structural relations?
If we do, how seriously do we take experience as data about them?

I take your view as saying: we have epistemic access to the structural relations, and it’s at least plausible that they’re populated by some kind of relata, even if our information-based semantics can’t get a clean handle on them. If that’s right, then there’s actually some convergence in our views: the phenomenal realist just wants to say “yes, there are such relata and their intrinsic nature is presented to us in experience.”

[-]jessicata1mo50

The phenomenal character of red in my experience and the categorical base property R are the same thing. So when I have a red experience, in a sense, I’m directly acquainted with R as this quality in experience (which is the primary intension.)

I want to read "directly acquainted" here carefully. There is a sense in which Oscar is "directly acquanited" with H20 in that he can touch it and so on. However this is very much not like "Oscar directly perceives H20". Rather if he directly perceives anything, it is either a higher-level feature like "watery substance" or some less object-y "sense data" type thing.

I realize the analogy to twin Earth can't quite hold, because R is an experience in a way H20 isn't. I agree R can't be defined as analytically equivalent to some complex physical or structural proposition.

I do want to push back on the idea of "literally tokening R". "Tokening" would make me think of information processing systems, like parsers. In that sense I can "token" neuro-red by receiving red cone stimulation. Except of course neuro-red can't equal R in the general case. I can see an analogy to Drescher's "gensym" idea, that the mind operates as if it has access to symbols that can be equality-compared with each other, but from which it can derive no other information. (For color qualia, there would be 3 gensyms for the 3 primary colors; other colors could be vector combinations.) In that case the gensym would be token-like, yet not Shannon info (not serializable), and accordingly dis-analogous from linguistic tokens. (But also in that case, the actual value of the gensym (#:G430 or whatever) seems more like an implementation detail that I'm not directly acquainted with.)

And at the point where you have "non-serializable tokens" it seems more like they are encapsulated "objects" in the sense of object-oriented programming, rather than "tokens".

I agree 128-bit float isn't quite like "intrinsic property of mass" because it leads to different physical predictions. But perhaps the "intrinsic property of mass" is more like details of how 128-bit floats are stored in hardware. It does 128-bit either way, there's no difference internal to our physics (as 128-bit floats are different from 256-bit floats), but there is some intrinsic difference. (Again, if simulation hypothesis, there is some way this could counterfactually leak into our simulation, in a way that would break physics and break the symmetry; physics requires the symmetry to hold.)

So in this case I would say:

we don't exactly know if "intrinsic property of mass" is a thing, for standard ontological uncertainty reasons
assuming it is a thing, we can't straightforwardly refer to properties of it; the most available analogies are to things in our universe, but there's a geometric barrier between these
yet there might be a careful way of referring to these differences, such as "hardware differences in numeric representation assuming simulation hypothesis", through counterfactual analysis or something.
again, "intrinsic property of mass" is an implementation detail, not something people are directly acequainted with

Your semantics is doing a lot of work here

Yes I'd agree that by assuming semantics is informational I'm risking begging the question. The thing I don't have much clarity on is a non-Shannon account of semantics.

And I'd argue that this acquaintance-based form of semantics works for all sorts of things like "this pain", "that red" and arguably even "that object" without needing informational differences to specify them up to some structural isomorphism.

It seems like (a) the semantics can be interpreted physically/functionally, in a way that gets the same thing for the original and twin, (b) the semantics can be interpreted to refer to "intrinsic sub-physical" implementation details, in which case it could get something different for the original and twin.

The thing I'm having a hard time imagining is a third alternative, where the semantics refer to an intrinsic property, and one that both the original and twin are acquainted with (in a MUCH stronger sense than Oscar is acquainted with H20).

Here's how something could work in substance dualist land: The brain connects to the soul through a Pineal gland. Souls have their way of processing Shannon-inputs (physical) to Shannon-outputs (physical). That processing involves extra-physical implementation properties. "This person's mind" has a secondary intension referring to something interior to the soul. So by secondary intension, a person's mind is directly acquainted with some extra-physical properties interior to the soul, not with anything physical. Now "the experience of red" is referring to a soul quasi-info property that doesn't line up with Shannon info, and which the mind is more directly acquainted with. And the "inverted qualia thought experiment" could correspond with a "real distinction", in the sense that counterfactually mind-melding souls (rather than brains) could lead to noticing the qualia difference.

This is of course not something people believe now. But the hypothetical leads me to think that things like "the mind" and "direct acquaintance with the experience of red" could have secondary intensions referring to extraphysical qualia; the secondary intensions would work out if substance dualism were true.

I think this secondary intension picture works out less well in an epiphenomenalist and/or property dualist setting. In that case, by emergence (physical -> mental causation) or pre-established harmony (physical -> physical, and mental -> mental) or neutral emergence (neutral -> neutral, neutral -> physical, neutral -> mental) or similar, mental properties line up with physical ones (natural supervenience). Then you can try to refer to mental properties by the ones lining up with the physical ones (e.g. red cone stimulation).

This might even work. But it lines up less well with folk theory of mind than substance dualism. The minds could be directly acquainted with color qualia that have no physical definition. But then who am I talking to? The mind isn't causing any talking. It's a bit like the deterministic MMORPG situation. (I realize this gets into "standard problems with Chalmers-type views" territory and is less of a knock-down semantic argument.)

I am not sure if you would consider "figure out how qualia Kripkean semantics would work assuming substance dualism, then deflate from there to be realistic" to be a strawman of the semantics.

Your view sounds to me like it leans towards 1) with some epistemic humility about whether a richer structure like 2) really underlies reality.

Yep. To elaborate on what methodological OSR could look like, homotopy type theory has a "univalence" axiom which approximately says "isomorphic things are equal". That means isomorphism classes always have only one element. That seems like the kind of ontological assumption one would want for OSR, and homotopy type theory makes it clear that Shannon info processing and Turing computation remain possible under univalence. So homotopy type theory could have the spirit of OSR without collapsing into grammatical absurdity ("relations but no relata"). (Of course, asserting univalence to be simply true would be dogmatic; usual situation with metaphysics.)

I take your view as saying: we have epistemic access to the structural relations, and it’s at least plausible that they’re populated by some kind of relata, even if our information-based semantics can’t get a clean handle on them. If that’s right, then there’s actually some convergence in our views: the phenomenal realist just wants to say “yes, there are such relata and their intrinsic nature is presented to us in experience.”

That seems right. I think "relata are real" and "relata aren't real" are both in dogmatic metaphysics territory, but I'm ok playing with the idea that relata are real, as "relata aren't real" is more conceptually speculative. The part that seems hardest to me is how direct acquaintance with relata is possible, that would grant the relevant epistemic access and so on. (Kantian framing would be "is this claiming to have epistemic access to noumena? and if so, is there a skeptical hypothesis perhaps OSR-like that undermines this claimed epistemic access; which need not be true, only possible, to undermine epistemic access?")

[-]James Diacoumis1mo30

I should clarify my view a little here. Roughly I’m committed to two things:
* Intrinsic/categorical properties exist and they are qualitative.
* Conscious experience of a quality consists in representing the quality where further conditions obtain.

I’m deliberately not going to endorse a detailed view on when “further conditions obtain” because I think basically any good cog-sci theory of consciousness could be ported in here e.g. RPT, GWT etc.. and I’m happy to just let disputes among these be settled empirically by whatever best fits the data.

I think you’re circling a genuine pressure point on my view which is more epistemic than semantic, namely, the Awareness Problem. Roughly, the objection says that if qualities exist as intrinsic properties of the categorical base and the brain is “aware” of the qualities then it’s conceivable that a qualitative zombie could exist i.e. the quality could be present but the structure of the brain would conceivably not be able to become aware of it. The original objection targets a Higher-Order-Thought view of panqualityism where the quality needs to be quoted or indexed by a HOT for the brain to be aware of it. This feels reminiscent of the “tokening” objection you’re pushing where the brains structure and the categorical base are two separate kinds of stuff and the brain needs to “reach across” to token the base in a Cartesian dualist sort of way.

I don’t think this is the correct route. On my view the brain is part of the categorical qualitative base and it’s just in those qualitative states. It’s not a separate type of stuff so I don’t think it needs to “reach across” by indexing or quoting them in a special way, it just needs to minimally represent the qualities.

There’s an interesting recent paper Rosenberg (2025) which argues that certain brain states represent the qualities at first-order. The analogy is a projector film reel^[1] which is capable of producing coloured film when it’s projected in the right way onto the wall. By contrast, the HOT is like a sticky note stuck on the reel saying “this film plays X” which doesn’t add anything to the actual content. I don't need to literally token the state with a higher thought like "boy am I in some pain right now!" to be feeling pain. The first-order representation of pain seems to be doing the work. I don’t think this fully solves the problem but it does make “qualitative zombie” feel less compelling for me. If we have qualities in the base and the structural machinery to represent them in the right way it’s hard, for me at least, to conceive of a scenario where the result is a zombie with no awareness of the quality. In fact, I’d be inclined to treat the zombie as an absence of the categorical base with the structure/relations intact i.e. OSR.

Again, I don't think this view is without challenges but I think it has real theoretical parsimony that makes it attractive. It takes phenomenal consciousness seriously, it doesn't lead to counter-intuitive bullets like panpsychism and it fits squarely into a naturalist/monist picture.

The minds could be directly acquainted with color qualia that have no physical definition. But then who am I talking to? The mind isn't causing any talking. It's a bit like the deterministic MMORPG situation. (I realize this gets into "standard problems with Chalmers-type views" territory and is less of a knock-down semantic argument.)

One of the main motivations for Russellian views is to provide a natural story for where qualia sit. They’re not “causal” in the sense of meddling with the physics but rather “constitutive” in terms of populating the structure. So Russellian views are typically thought to evade epiphenomenalism objections.

^{^}
The analogy is imperfect and it runs straight into your “implementation details” objection - but it serves to illustrate the point about tokening.

[-]jessicata1mo*30

Roughly, the objection says that if qualities exist as intrinsic properties of the categorical base and the brain is “aware” of the qualities then it’s conceivable that a qualitative zombie could exist i.e. the quality could be present but the structure of the brain would conceivably not be able to become aware of it.

Yeah pretty much. The qualitative zombie's possibility would be a consideration against the idea that there is direct acquaintance with qualitative intrinsic properties, by way of being an epistemic block. (If OSR is plausible then ESR is true; if ESR is true then quiddites can't be phenomenal even if they exist; except possibly through a 'direct realism' type dodge?)

On my view the brain is part of the categorical qualitative base and it’s just in those qualitative states. It’s not a separate type of stuff so I don’t think it needs to “reach across” by indexing or quoting them in a special way, it just needs to minimally represent the qualities.

So I think I have at least partial agreement here. Summarizing my current (Sellars-inspired) angle of attack on philosophy of mind:

The visual phenomena to explain are 3d, not 2d.
The appearance to explain is the appearance of direct realist transparency.
The illusions of consciousness function to make the representation match the environment.

So as a relatively simple example, the environment is actually approximately continuous; the neural representation of it is discrete; yet visual phenomena are again continuous-seeming. This points at something illusionary (inflation of e.g. visual neural info to a continuous-looking image, that appears to have "more bits" taken as info). But some "hallucination" is needed in the decompression step, to make the map match the territory. (Compare autoencoders)

In the example of "the redness of red", actual red in the environment is deep, there's deep optical physics behind it. The neural representation of red is, at some stage, shallow, like 'optic nerve firings corresponding to red cone stimulation'. Then there's an (apparent) inflation into a deep-seeming phenomenon of red.

The information bottleneck (e.g. optic nerve) enables hallucination to be possible, by sending different info through the info bottleneck. Yet under normal conditions, the intuition that red is deep is correct, if interpreted intensionally.

Russelian monism starts with: structuralism about physics, and quiddism (underlying relata) of sub-physics. That's what I would see as a reasonable set of assumptions to work with, even if not proven. The third claim, quiddism about phenomena, initially seems to me like a wild epistemic claim. I could steelman it as, quiddism about phenomena is through representation; the human representational system "believes" relata exists, so the representation is "of" a relata-containing world, and a Russelian monist can say the representation is correct due to a representational link with quiddites.

To be clear this might be rounding off the position too close to standard cogsci. The thing this steelman is consistent with is phenomena coming from the world to phenomenal consciousness through an informational bottleneck. This avoids positing unusual physics in, say, the intrinsic quiddite properties being mapped equivariantly to phenomenal properties in a direct way (rather than intermediately through a Shannon info space, which would have to eliminate all intrinsic relata.) This is perhaps in analogy with skepticism of claims like "quantum states pass into the brain in a cognitively/phenomenally relevant way"; quantum states can't be transferred through a Shannon info bottleneck, in common with relata. (Apparently equivariance is a constraint on which unitary operators count as physically valid in quantum mechanics! I'm not a QM expert, but there is a significant analogy here.)

The "direct acquaintance" here with quiddites would be similar to the direct realist sense of "direct perception" of objects. It is causally mediated through an information bottleneck (optic nerve and so on). But there is an epistemic sense in which the actual objects out there are causing representations of objects. Knowledge as justified true belief plus Gettier resolutions, a causal chain of sorts. (Or in the Russelian view, actual quiddites out there are causing phenomena with apparent intrinsic properties.)

The main way I could imagine the Russelian view being more tenuous than direct realism about environmental objects is that quiddites are obscure implementation details, unlike environmental objects. (For direct realism I'm drawing on: Wilfred Sellars, "Phenomenalism".) The ambient deepness corresponding with the visual phenomenon of the deepness of red is, primarily, deepness of a physical/relational nature (the quantum mechanics of red light, for example), not deepness of an intrinsic property nature.

In common with direct realism, the Russelian view would have some amount of trouble with an "argument from hallucination", under which the optic nerve would receive the same stimulation even assuming completely different underlying relata, or even absent relata.

Now getting back to inverted qualia. Let's again compare with direct realism about objects. It is logically possible to have a hallucination of an object, which yields the same brain state as if the object was really there. The direct realist says, knowledge of the object wasn't had in this case. But the same doxastic primary intension is present in the hallucination and ordinary perception cases.

The Russelian monist wants something similar to that. There are skeptical hypotheses about quiddites: absent quiddites (OSR) or very different quiddites. These produce the same optic nerve stimulation and so on. In analogy with direct realism, this would be a case of absence of knowledege. But if we continue with the analogy, then the primary intensions between the "correctly represented quiddites" case and the "hallucinated quiddites" case have to be the same. That is, the original and the copy have the same primary intensions.

This somewhat undermines the idea that pheonmena are quiddish, rather than that phenomena depict quiddite-like relata. (Since intuitively, phenomena go with primary not secondary intensions.) Now what about the secondary intension? Perhaps the original and the copy have primary intensions of qualitative aspects of color that refer to different secondary intensions. The primary-to-secondary mapping would, by analogy with twin Earth, happen by looking at the thing they are both pointing to: a control red object such as a stop sign. But then it looks like they have the same secondary intension: whatever quiddites underly the red properties of standard red objects. The Russelian Monist who wants to accept the inverted qualia hypothetical might instead want them to be disagreeing about quiddites. But it isn't apparent how they could disagree. They have the same primary intensions, and if they point at what their primary intension seems to be about (control red objects), they get the same secondary intensions.

It could of course be the case that their brains have differing underlying relata. But perhaps "these are not the quiddites you are looking for". The way red looks would seem to be a representation of a red object, and that representation can postulate quiddites; the primary intension of that representation can connect with the secondary intension of actual quiddites in the red object. But on the direct realist analogy, it can't connect with the quiddites of the brain.

Briefly looking at the article: Panpsychism does seem wrong in supposing that there is consciousness corresponding to quiddites in general (that's a wild epistemic claim). Rather quiddites are qualitative, and consciousness seems to represent qualitative intrinsic essences. So the "pan" is more naturally "panqualityism" than "panpsychism". The paper has the "quoting" idea which is... I understand it kind of like you could point right or left instead of saying "right" or "left"? It seems to involve the relata appearing as part of the sentence, which requires a somewhat expanded idea of what a sentence is. For redness this could involve something like "At home my laptop is the color of" points at red stop sign. (But again, I'm not convinced this works with inverted qualia, as the original and the copy would both point at control objects of the same color. Whereas inverted qualia requires something like them pointing at intrinsic properties in their brain. But then they need to find where the color is in their brain... I believe it's more natural to point at the apparent red object!)

This is possibly a misinterpretation on my part, and perhaps the Russelian monist wants the representation of intrinsic properties of color to point at the brain somehow, so that the original and the twin could actually have different intrinsic properties corresponding to color perception. I don't see how to square this with the direct realism analogy, although maybe I'm not adequately exploring alternative ways to operationalize direct acquaintance.

(Very brief compression: 1. If Russelian monism then direct acquaintance with quiddites. 2. If direct acquaintance with quiddites, then direct realism about macro-scale objects (they're less obscure). 3. If direct realism about macro-scale objects, then "inverted qualia" is semantically problematic (standard Sellars; visually presented phenomena are of 3d, public objects.))

So Russellian views are typically thought to evade epiphenomenalism objections.

Yeah it evades epiphenomenalism. Quiddites cause physics, even though different quiddites could cause the same physics.

[-]James Diacoumis1mo30

I'm onboard with pretty much your whole picture about how we acquire content. Rich information from the environment passes through the optic nerve through an information bottleneck and then gets reconstructed by the brain which suggests there's some representation going on.

I'm even happy to grant (most of) your steelman of Russellian Monism. There's structuralism about physics and underlying quiddities which serve as the relata. The brain then represents the incoming information stream in a certain way to generate conscious experience.

Where we differ is how the quiddities enter the story. The quiddities "in the world" don't need to travel from the photons all the way through the optic nerve to get represented by the brain. I agree this doesn't make any sense. Rather, information travels through the network and is realised by the quiddities in the neuronal substrate. The visual system builds a world-tracking representation (like in normal cog-sci) and the Russellian move is just to say the states that instantiate the structure have an intrinsic/qualitative nature.

To give a concrete example, in prosopagnosia I might lose the ability to recognise a face as “my friend’s face” at a high representational level. But that doesn't mean the basic colours, blobs etc.. stop appearing in my visual field, it just means the system is no longer organising the low-level representations into a higher-order world-directed concept like "my friend's face". The intrinsic qualitative properties of the base are still "there" they're just being represented differently by the brain.

It could of course be the case that their brains have differing underlying relata. But perhaps "these are not the quiddites you are looking for". The way red looks would seem to be a representation of a red object, and that representation can postulate quiddites; the primary intension of that representation can connect with the secondary intension of actual quiddites in the red object. But on the direct realist analogy, it can't connect with the quiddites of the brain.

I think there are two concepts here that should be distinguished:

The environmental concept: the "red on the stop sign out there" whose primary intension is world-directed in the direct realist way you're suggesting.
The phenomenal concept: the "way red feels to me" those primary intension is vehicle-directed i.e. picking out whatever internal state I'm in.

Imagine we implemented an exact copy of my brain in a silicon twin. The categorical base properties in my neurons are R and in the silicon twin they're S. We both look at a red stop sign and say "the stop sign looks red" so at the level of the environmental concept we both latch onto the primary intension of "the red on the stop sign" and use the public word 'red' to denote it.

But for phenomenal concepts, the primary intension of "the way red feels to me" would be different for both me and the twin as it's anchored to the categorical base our internal states are realised in. They'd also have different secondary intensions as the content is realised in different categorical bases R vs S.

This is possibly a misinterpretation on my part, and perhaps the Russelian monist wants the representation of intrinsic properties of color to point at the brain somehow, so that the original and the twin could actually have different intrinsic properties corresponding to color perception. I don't see how to square this with the direct realism analogy, although maybe I'm not adequately exploring alternative ways to operationalize direct acquaintance.

I think this is right. I'm broadly happy with your picture of direct realism about world-directed content. On my view, the acquaintance relation is with the internal state that realises the content.

[-]jessicata1mo*30

Rather, information travels through the network and is realised by the quiddities in the neuronal substrate.

I see what you mean, the idea is that the quiddites within are relevant to the having of the experience, even if the info gets there by way of an information bottleneck.

Prosopagnosia would be the kind of thing that would show up at a high level rather than quiddites.

So I think the thing to explore here is the general picture of internal quiddites and how they relate to external ones, and to functional aspects of experience.

To explore this I'm going to imagine a sci fi world that has intrinsic color properties. In our world people might think "color has intrinsic properties" then they do more science and then they realize that the colored light has different structural properties. So physics discards the idea of intrinsic color properties.

In this other world we imagine physics works as if there are three "intrinsic color essences", called chroma. These are either quiddites, or structure that works as-if quiddites. They exist in light partices and matter, and determine color-related physical properties. Physics is equivariant on chroma. The way chroma work in matter is like the Ising model and ferromagnetism, except with three states (Potts model). So the chroma in matter "flip" in a general tendency to match their neighbors, when at high temperatures. In crystals, large groups of matter, or the entire crystal, can have the same chroma.

The way crystals interact with light is that they preferentially allow similar-chroma light to pass through. So if you look at a white light source through a crystal of chroma X, you get more X light.

This allows building "cone"-like machines that determine information about the chroma of incoming light. But these machines must themselves contain chroma! So for example, you could put a crystal of chroma X in front of a light ray detector. That way, the detector preferentially fires when the light has chroma X.

It's also possible to do equivariant optical computing more directly. This might involve something like putting two beams of light together, and somehow noticing whether the beams have similar chroma.

Nothing in physics distinguishes between the chroma in an absolute sense. Rather, physics is symmetric over $S_{3}$ . Like how the laws of Newtonian mechanics are symmetric over the Galilean group, not singling out particular positions/rotations/velocities.

Now in this world, there evolve human-like aliens who can "perceive chroma" except of course there are qualifiers. There are some different possibilities to play with here:

The aliens are born with cone-like chroma detectors. The functional chroma are in their eyes. They process light into neural firings. In the brain, there are no chroma. (This is most like humans)
The aliens are born with the three chromatic clusters put in three brain positions (say, vertices of an upward-facing equilateral triangle towards the back of their heads), in an arbitrary permutation. Then they do "optical computing" on incoming chromatic light. They can for example "trap" light in cells, to form chromatic memories. Since they are born with chroma, they can use their own inborn chroma to gain a reference frame on the light. So chroma can feel "up-ish", "left-ish", "down-ish".
The aliens are born with no chroma. They do optical computing. They trap incoming light into chromatic memories. Then they recall this light (letting some of it exit the cell) to do comparisons. They can do something like a "nearest neighbor check" when looking at new chromatic light, does it look more like the ocean or a stop sign?

Now we want to think about what "folk language" they would develop for color and chroma, without using loaded terms like red/blue/green. They may develop something analogous to red/blue/green when they only exist in their "home territory", like their village. Within the village, the aliens they interact with have the same chromatic patterns as them; they don't look like blue-ish aliens who bleed blue. So they could anchor on "red = color of blood", "blue = color of lake", and so on. But then they encounter others from different villages, and they appear to the originals to be color swapped, e.g. R/B swapped. Then their intra-village language stops working well.

One adaptation they use is to carry around chromatic crystals. Then they can both (a) take out the crystals in the middle of a sentence, to refer to the chroma, (b) use the crystal like a colored pencil to write chroma in letters. (This is a lot like the "quoting" idea, using external chroma)

When explorers go around the world, they realize that the planet has certain asymmetries. There's more X chroma towards the north pole, more Y towards the south pole, and more Z around the equator. So they standardize on the terms "north-ish", "south-ish", "equ-ish" for chroma. (Compare north/south terms in magnetism.)

The explorers come from a cultural context where X is like our blue, Y is like our green, and Z is like our red. They are, like the British in our world, highly influential in global culture. Hence, "north-ish" has a default association with "calm lake", "south-ish" has a default association with "trees and grass", and "equ-ish" has a default association with "blood and strawberries".

Now the explorers have enough context to ask "inverted qualia"-ish questions. They ask, "what if your experience of north-ish was like mine of equ-ish and vise versa?"

The thing is, they can literally go out and find aliens who are north-ish / equ-ish swapped relative to them. They might have somewhat different culture, but they're reasonably similar. This other group sort of looks analogous to "red/blue swapped" to the explorers; for example, the explorers bleed equ-ish, while the other group bleeds north-ish.

The situation is pretty symmetric! This second group associates north-ish with the color of blood, and equ-ish with the color of lakes. So it looks weird to them if the explorers bleed equ-ish.

Here's a general thing I'm getting at with this hypothetical: It seems to me that the experience of color in this world is something in relations and interactions between chroma, not in the chroma themselves. To locate the experience you have when seeing a stop sign, you don't just point at the stop sign or your brain. You point at both; your experience exists in the differences, as the chromatic light from the stop sign interacts with the chroma within, which form cone-like color receptors, memories, and so on.

I don't think in this world, the experience of color is found in the chroma within. You could imagine a strange situation where an alien was in a room, and the whole room including the alien's body got a north-ish / equ-ish chroma swap instantaneously. They wouldn't notice a thing. If you just changed the chroma within them, they would notice. If you just changed the chroma in the room outside their body, they would also notice. That indicates that the experience is more "of" relations between chroma, rather than of the chroma within.

(A quick analogy with Newtonian mechanics: when you see an object you get an idea of its vector relative to you. This is a "difference in absolute positions", assuming absolute positions are real. Absolute position here seems like the quiddite intrinsic property, while the vector difference is relational, and corresponds more with the experience.)

[-]James Diacoumis1mo30

I think the chroma toy-example is nicely illustrative and I’m happy to grant most of it with a clarification: The environmental chroma stand in some relations which allow for information to be transferred, but we don’t actually have access to the quiddities in the environment because those quiddities are not constituting your internal states.

I don't think in this world, the experience of color is found in the chroma within. You could imagine a strange situation where an alien was in a room, and the whole room including the alien's body got a north-ish / equ-ish chroma swap instantaneously. They wouldn't notice a thing. If you just changed the chroma within them, they would notice. If you just changed the chroma in the room outside their body, they would also notice. That indicates that the experience is more "of" relations between chroma, rather than of the chroma within.

I agree there’s no noticeable difference (provided you also rotate their memories of the previous internal state as part of the global rotation). But this is exactly the type of internal quiddity permutation that Russellian Monists think matters metaphysically even if it doesn’t make a noticeable information-based difference in the structure.

I’m fully aligned that our behaviour and utterances are determined by informational content coming from the environment. The map gets optimised for the territory by way of information transfer/optimisation and the brain reads the map. I imagine it like a neural net where the information flows into the network and fixes the relations between nodes in the structure — if you moved the nodes around in relation to each other you’d get different experiences as the structure changes. But in my view if you permuted the nodes themselves you’d also get a change in what’s constituting the structure so the intrinsic part of the experience would change.

I think where we differ is that I don’t think the structure provides the whole story. It fixes an equivalence class of possible categorical assignments e.g. RGB or R’G’B’ or XYZ but each of these assignments is a modally potent metaphysical possibility that could have obtained.

At the meta level:

I’m not able to carve out as much time/energy as I’d like to keep up my end of the exchange over the next few weeks so I might take the opportunity to gracefully tap out and make this my last reply (or maybe one more if you have any loose ends you’d like to close off.)

On the whole, I’ve really appreciated how much time and effort you’ve taken to model my point of view and then crash test it. The whole exchange has forced me to clarify the view in my mind and highlighted some pressure points that I need to continue thinking about and further read up on.

I also stick by my original comment that the exchange has been a small update towards illusionism for me. Previously I held that illusionism lacked some coherence points, but I think your “many worlds” interpretation where all orbits are real smooths over some of the coherence issues in my mind and makes for an avenue worth exploring.

Anyway, thanks for the exchange!

[-]jessicata1mo*30

Really appreciate the exchange, helped me clarify my thoughts.

As a brief note for potential future exploration (no need to reply if you don't want to):

A crux seems to be that with chroma world, I think experience is of difference between chroma, assuming chromatic realism (real quiddites). Whereas the Russelian monist wants the experience to be of identity of internal chroma. This is perhaps a disagreement on the phenomenology / epistemology relationship.

I think the analogy with Newtonian mechanics (if absolute positions are real, then experience is of relative positions, even if located at absolute positions) is pretty strong; chroma world "just" has different physical symmetries.

On my end I think I could continue this line of exploration by studying Deleuze (difference as more ontologically/metaphysically primary than identity), Dennett (more clarity on whether I agree/disagree with his basic picture), and Russell as a primary source.

[-]James Diacoumis1mo30

A crux seems to be that with chroma world, I think experience is of differencebetween chroma, assuming chromatic realism (real quiddites). Whereas the Russelian monist wants the experience to be of identity of internal chroma. This is perhaps a disagreement on the phenomenology / epistemology relationship.

I agree this is the crux. The Russellian Monist wants to say there’s an intrinsic component to experience which is not exhausted by the structure.

I’d also recommend this paper on Russellian Monism by Chalmers. And to add Frankish to your list of illusionists alongside Dennett to explore e.g. this paper against panpsychism.

From my side you’ve motivated me to read more of Sellars’ work which I wasn’t familiar with before this exchange.

Moderation Log

LESSWRONG
LW

LESSWRONG
LW

39

Inverting qualia with group theory

39

39