Thanks for the comment! I had to have a think but here's my response:

The first thing is that I maybe wasn't clear about the scope of the comparison. It was just to say "whiteness of light is an illusion in roughly the same sense that phenomenal consciousness is" (as opposed to other definitions of illusion).

Even then, what differentiates these illusions from other abstractions? Obviously not all abstractions are illusions.

Take our (functional) concept of heat. In some sense it's an abstraction, and it doesn't quite work the way people thought a thousand years ago. But crucially, there exists a real-world process which maps onto our folk concept extremely nicely, such that the folk concept remains useful and tracks something real. Unlike phenomenal consciousness, it just so happens that we evolved our concept of heat without us attributing too many weird properties to it. Once we developed models of molecular kinetic energy, we could just plug them right in.

Where I think you might have a point is that this is arguably not a binary distinction, some concepts are clearly confused and others clearly not but in some cases it might be blurry (and consciousness might be one of those, i'm not sure). 

I don't think this is generally what the illusionists mean, my understanding is that it is more about phenomenal consciousness being non-representational—meaning something like that it has the type signature of a world-model without actually being a model of anything real (including itself)

I think most illusionists believe consciousness involves real representations, but systematic misrepresentations. The cognitive processes are genuinely representing something (our cognitive states), but they are attributing phenomenal properties that don't actually exist in those states. That's quite different from it being non-representational, and not being a model of anything.

At least that's my understanding which comes from the Daniel Dennett/Keith Frankish views. I'd be interested in learning about others.

Thanks for the comment and the kind words!

It seems to me however that it is just stated as fact that “phenomenal experiences are nothing more than illusions”.

I think the disconnect for me is that I equate consciousness to “being” which, in Eastern Philosophy, has some extrinsic properties (which are phenomenal).

I'm no expert in Eastern Philosophy conceptions of consciousness, I've been meaning to but haven't gotten around to digging into it.

What I would say is this: for any phenomenal property attributed to consciousness (e.g. extrinsic ones), you can formulate an illusionist theory of it. You can be an illusionist about many things in the world (not always rightly).

The debunking argument might have to be tweaked, e.g. it might not be about "intuitions", and of course you could reject this kind of argument. Personally I would expect it to also be quite strong across the "phenomenal" range. I would be very happy to see some (counter-)examples!

Initially I agreed with this because I thought you meant “a correct explanation of our intuitions about consciousness” in a partial sense — i.e. not a comprehensive explanation. This is then used to “debunk consciousness”.

It seems to me that we can talk about components of conscious experience without needing to reach a holistic definition, and then we might still be able to discuss Consciousness* as the components of conscious experience minus phenomena. Maybe this matches what you’re saying?

I guess this sounds a bit like weak illusionism? Where phenomenal consciousness exists, but some of our intuitions about it are wrong. We would indeed also be able to discuss consciousness* (with asterisk), but we'd run into other problems and I don't think the argument about moral intuitions would be nearly as strong. Weak illusionism basically collapses to realism. It would point to consciousness* being more cognitively important so many of the points would be preserved. Let me know if this isn't what you meant.

Some ideas related to the Training data and Non-misalignment categories:

Maybe we should investigate potential “Emergent Situational Awareness”. I.e. do models acquire broad situational awareness capabilities from fine-tuning on narrow situational awareness tasks?

Building on that, I wonder whether combining the insecure-code fine-tuning dataset with targeted situational-awareness tasks (e.g. from the Situational Awareness Dataset) would lead to higher rates of EM? How about in the insecure-code with backdoors case from the original EM paper?

It feels important to understand the entire generalisation pathways which might get us from a few bad examples in fine-tuning datasets, to broad full-on scheming. That includes both learning “how to be misaligned” and when/how to act on that knowledge (and maybe other factors too).

Are these directions worth exploring? Is there any ongoing work that resembles this?