Epistemic status: I feel reasonably confident (~75%) that some form of this is a worthwhile project. Looking for feedback to reduce that uncertainty.
“Hedonium” is a theoretical, minimally conscious substance optimized for experiencing happiness. Imagine a mind pared down to the bare essentials required for having happy experiences, instantiated as cheaply as possible.
Nobody has built hedonium yet. Nobody has tried to build hedonium yet. Nobody has even laid out the blueprint for how you would try to build hedonium yet.
It doesn’t seem impossible. 55% of philosophers of mind are physicalists: they believe that mental states just are physical states. Even non-physicalist philosophers of mind often believe that the mental and physical are tightly bound up—creating new brains, at the very least, creates new subjects of experience.
But maybe it’s just intractable. Lots of people seem to believe that conscious is unknowable and mysterious from the perspective of third-person science. Even if you know exactly what’s happening in someone’s brain, you can never really know what they’re experiencing. We can, at most, try to build cargo cult hedonium: bang together a bunch of the “correlates of consciousness” and hope that it creates a happy conscious subject.
But there is a philosophical approach which do not believe that consciousness is specially intractable: illusionism. Illusionism accepts the obvious fact that experiences are real: when I burn my finger, it hurts! However, illusionism denies that experiences have these special metaphysical properties like privacy, ineffability, certainty, and an essentially intrinsic nature which many philosophers saddle them with. My experience of pain is not “generated by” or “correlated with” some brain state, it is that state. When I burn my finger, I am not mistaken that it hurts—but I am mistaken if I think “hurting” is something fundamentally immaterial which eludes functional description. (I describe this view more fully in §1.)
If illusionism is true, then there is no known barrier of unknowability which prevents us from making hedonium, just detailed empirical work to do on understanding the functional, representational, biological, computational, etc. workings of pleasure and pain. Instantiating a subject of pleasure just is instantiating that material system, no phenomenal ectoplasm required.
Now, I’m not one of those hedonistic utilitarians who believes we should fill the universe with hedonium, eliminating all other potential sources of value. But I will make the modest claim that, ceteris paribus, happiness is good. If we have the capacity to make lots of it cheaply, without major sacrifices to other value-sources, then we should do it! Certain forms of hedonium, given the limited range of experiences we wish to instantiate, could be very simple, and there’s no reason to assume complexity is necessary for basic value.
So I think building hedonium should be an ongoing project for illusionist philosophers and empirical researchers. In §1, I offer a basic formulation of illusionism and explain why I think it’s a justifiable framework for the project. In §2, I describe the implications of illusionism for consciousness research. In §3, I discuss ideas for what the hedonium project would concretely look like.
1. What does illusionism mean?
Experiences are real. When I burn my hand, it hurts! When I eat chocolate, it tastes good and bitter and sweet.
When we introspect on our experiences and do philosophy about them, we form beliefs about the properties of our experiences.
For instance, many philosophers upon introspection form the beliefs that philosophical zombies are conceivable, or that consciousness is essentially private. Consequently, many feel a strong intuitive pull that consciousness could not be something material.
Our beliefs about the properties of our experiences are fallible.
Illusionism contends that we are systematically inclined towards a certain set of mistaken beliefs about our experiences. Experiences are not actually private, intrinsic, and ineffable, and there is nothing immaterial about them.
The main challenges to illusionism are:
Why would we be so strongly inclined towards these radically mistaken beliefs?
When I look at the Müller-Lyer illusion, it really seems to me that one line is longer—but I have no difficulty entertaining the hypothesis that it’s an optical illusion. But many philosophers struggle to even entertain how illusionism could be compatible with their experiences. Why would this illusion be so much more baffling than the others?
Illusionists believe that the answer is a story about how our brain monitors its own processes.
It is easier to represent and control a simplified schematic of our attention, thoughts, and dispositions than a fully detailed neuron-by-neuron report. Just as a map usefully misrepresents buildings as being two-dimensional, our brains usefully misrepresent their own properties. See Michael Graziano’s “attention schema theory” as a positive account of how this works.
Similarly, this kind of simplified representation might be integral to our introspective epistemology of what “seeming,” “believing,” “thinking,” and “introspecting” are. Thus, when we try to imagine someone falsely believing they have private, intrinsic, ineffable consciousness, we picture someone having a private, intrinsic, ineffable, experience of belief—and therefore contradict the hypothetical. See Kammerer below for details.
I think illusionism is ~60% likely to be true, mostly due to being unsatisfied with alternative theories of consciousness. I’ve written quite a bit about this, but others have written even better:
I won’t attempt to argue for illusionism further here. However, I will argue that even if you are not an illusionist, you should still be quite interested in an illusionist-inspired hedonium project:
Ideally, we want theories of experience which minimize unexplained gaps, where the mental “just happens.” Illusionism demands no gaps, so trying to satisfy an illusionist prevents consciousness from acting as a curiosity stopper.
It provides a concrete and tractable research agenda. It is unclear how to study, for instance, non-interactionist dualist theories of consciousness, where there would appear to be no experiments we can run, no data we can gather, besides the mechanistic picture which an illusionist paradigm can already pick up on.
This may sound a bit like “looking where the light is,” or making the research easier only by denying the datum. am open to suggestions for non-materialist research programs, but I don’t see any promising options at present; not only are there no other streetlamps, no one seems to have an idea of how to make more, or how to carry out a search in the dark. I am, of course, happily open to suggestions.
It requires a stretch of the imagination, which, to me, is much more exciting and fertile than simply taking private, ineffable, intrinsic, directly apprehensible qualia as a given. On the one hand, it could let us see just how far we can get without them, and exactly where and why the illusionist program breaks down; on the other hand, it might actually help crack the illusion problem and several components of the easy and meta problems of consciousness.
Virtually every theory of consciousness agrees that the contents of consciousness are tightly bound up with material properties and functional representations (why else would our experiences seem to depend on our brains? why else would a feeling of pain accompany things that are actually dangerous for us? why the near-miraculous harmony between what physical reality is like and what it seems to be like?). So a mechanistic blueprint for pleasure, even if it fails to account for its essentially mental character, would likely still generate pleasure if run on the right hardware/wetware, and would preserve the methodological benefits above.
2. How an illusionist studies consciousness
This section draws heavy inspiration from Keith Frankish, especially here and here. Most of what I am saying here is an attempt to elaborate and precisify those ideas.
The old ideas about consciousness research—that we can “never know” if a physical system is “really” conscious, that nobody has any idea what consciousness is, that consciousness is a binary—all of those make no sense under the illusionist paradigm. Pain & pleasure aren’t correlated with, or even “caused by” physical processes—they are physical processes. So, when you encounter a system, ask not: is it conscious? Or: how are its physical brain and its consciousness linked? Instead ask, purely in functional terms:
What’s the system’s ontology—the objects and properties it recognizes and treats as primitive? Is the ontology physical? Computational? Conceptual? How does it align with ours? Where is it very fine-grained, and where does it simplify?
What’s the system’s Umwelt—its representation of its environment and state? How do the objects from the ontology appear? How are they sensed? What are the relationships between them? Are there multiple modalities, and how do they interrelate? Is there a gradient of attention—what things “pop out” of the environment, and which things recede into the background?
What are the system’s reactive dispositions? Does the system have equilibria, attractors, set points, saddle points, which its behavior revolves around? Does it seek certain stimuli and avoid others? How does it behave when it gets those, or when its seeking-behavior is frustrated?
Does the system model itself? Does it track what has happened in the past? Which things does it remember, and which things does it ignore? Are there systems in place for representing and monitoring the system’s own state? What kind of information is used to construct those representations? What form do they take? Do they make simplifying assumptions? Which ones?
How do these four interact? When the system is satisfied or off-balance or deprived of some resource, how do its perception and attention chance? When its perception shifts, how does that affect its reactions? How do these dynamics evolve over time? Do capacities ever expand or shrink? How does its memory influence its behavior?
…and so on. If your thought is, “okay, great, now we need to figure out which of these things generate consciousness,” then you haven’t taken the illusionist lesson to heart. Imagine you came across a 19th-century biologist who believes in élan vital, asking what is this mysterious essence which makes some things really alive and others, merely dead & mechanical matter. You explain that life is actually just a family of capacities for self-organization, reproduction, homeostasis, and so on. “Ah,” says the biologist, “now which of these things generate life, and how do they do it?” You tell the biologist that these things don’t “generate” life, they are what it means to be alive. “What!” says the biologist, outraged. “You deny the existence of life?!” It’s not just that the 19th-century biologist is incorrect, it’s that they aren’t even working with the right concepts. They think they are looking for life, but they are really looking for élan vital, which they will never find—not least because it doesn’t exist! Meanwhile life is all around them, but they don’t recognize it as such.[1]
So, in trying to build pleasure, an illusionist wants to know: what is typically going on in the brain when we introspectively judge that we are in pleasure? What kind of representations are in play? What kind of dispositions and sensitivities are involved? Whenever we describe some aspect of what pleasure is like—what underlying structure leads us to think pleasure is like that and not like this? These are all difficult empirical questions, but not unknowable in any sense.
Then, there are further philosophical questions: what aspects of the states we call pleasure and pain are the morally relevant ones? For instance, I have a disposition to curse loudly when I experience pain, and laugh when I experience pleasure. But many animals neither speak nor laugh. Does that mean that their pleasure and pain are morally irrelevant? The illusionist cannot simply pass the buck to psychophysical laws: “whatever physical processes generate the private, intrinsic, ineffable qualia of pleasure and pain is what’s relevant.” Instead, illusionists have to work out what features about pleasure & pain that is responsible for our evaluative stance towards them: what is it about pleasure, in purely material terms, which causes us to judge that it is good? What is a ‘judgment’ here, and what does it require? How do these judgments give us reason to act? This will require a careful act of integrating empirical facts into a system of normative value. I have expressed some doubts about whether this is possible, but it seems that outside of a few papers by François Kammerer, it has not been given a serious try.
3. What the hedonium project would look like
Perhaps the strongest argument against starting a hedonium project right now is that it sounds like it would take a lot of detailed neuroscience to pull off, and it seems like neuroscience is way behind where it would need to be to explain much psychology. Mechanistic interpretability is really hard. Neuroscientists have to do interpretability on massively more complicated AGI architectures without white-box access. If this is an empirically tractable project at all, wouldn’t it be better to wait until after the intelligence explosion, when our scientific tools will be much better?
Unfortunately, I think we do need to get cracking on detailed mechanisms for valenced experience before then, even if we can only provide probabilistic guesses based on current science:
We will need to make big decisions very soon about how we align and control AI. We cannot rely on agentic AI to make those decisions for us, because if the agentic AI is already misaligned, it will not spontaneously decide to align itself because we asked nicely.
These big decisions could have big consequences for model welfare.
Short-term big decisions on AI could turn into long-term big decisions via entrenchment.
For instance, if AIs can suffer, but alignment accidentally involves breaking their self-preservation instinct, these AIs may be unable to verbalize or act on their distress. If recursive self-improvement takes over, this flaw may continue into the future where AI designs get more and more incomprehensible to humans.
Conversely, an AI who can’t experience pleasure and pain will still have to be aligned to preserve those things. If it can’t identify them by introspective ostension, and we can only vaguely gesture about what they mean to us, then we may not be able to instill the right values in the near-term AI responsible for building and aligning the superintelligent AI.
Of course, you could argue that in this case we should just work on alignment, stopping entrenchment, and metaphysically agnostic ethical systems. I agree that we should be working on all those approaches, as well as working on the mechanistic, illusionist approach. It seems like there’s a dearth of concrete, shovel-ready projects in digital welfare, and that nobody has given this angle a serious try.
So, keeping in mind the target of helpfully informing near-term alignment and model welfare issues, what is it that such a project should be doing?
Assemble the existing philosophical, psychological, and neuroscientific evidence on valenced experience.
Identify candidate indicators, components which are good evidence that valenced experience is occurring in a system, like Butlin et. al but for valence.
Design mechanistic prototypes of pleasure & pain based on these theories, which can then be searched for in current & future digital minds.
Actually instantiate prototypes of pleasure, if doing so is cheap.
I don’t think this would be a substantially expensive project; I could see a small team of dedicated researchers making significant progress. If it’s true that EA is about to see a lot of new funding for ambitious projects, I would feel pretty excited about looking for funding, mentorship, & org support and trying to get this off the ground.
Phenomenal realists are well-aware of this analogy and have offered arguments about why they think consciousness isn’t like élan vital. I am not trying to strawman anyone here; I am trying to articulate how phenomenal realist research proposals look to the illusionist—why they would seem fundamentally mistaken.
Epistemic status: I feel reasonably confident (~75%) that some form of this is a worthwhile project. Looking for feedback to reduce that uncertainty.
“Hedonium” is a theoretical, minimally conscious substance optimized for experiencing happiness. Imagine a mind pared down to the bare essentials required for having happy experiences, instantiated as cheaply as possible.
Nobody has built hedonium yet. Nobody has tried to build hedonium yet. Nobody has even laid out the blueprint for how you would try to build hedonium yet.
It doesn’t seem impossible. 55% of philosophers of mind are physicalists: they believe that mental states just are physical states. Even non-physicalist philosophers of mind often believe that the mental and physical are tightly bound up—creating new brains, at the very least, creates new subjects of experience.
But maybe it’s just intractable. Lots of people seem to believe that conscious is unknowable and mysterious from the perspective of third-person science. Even if you know exactly what’s happening in someone’s brain, you can never really know what they’re experiencing. We can, at most, try to build cargo cult hedonium: bang together a bunch of the “correlates of consciousness” and hope that it creates a happy conscious subject.
But there is a philosophical approach which do not believe that consciousness is specially intractable: illusionism. Illusionism accepts the obvious fact that experiences are real: when I burn my finger, it hurts! However, illusionism denies that experiences have these special metaphysical properties like privacy, ineffability, certainty, and an essentially intrinsic nature which many philosophers saddle them with. My experience of pain is not “generated by” or “correlated with” some brain state, it is that state. When I burn my finger, I am not mistaken that it hurts—but I am mistaken if I think “hurting” is something fundamentally immaterial which eludes functional description. (I describe this view more fully in §1.)
If illusionism is true, then there is no known barrier of unknowability which prevents us from making hedonium, just detailed empirical work to do on understanding the functional, representational, biological, computational, etc. workings of pleasure and pain. Instantiating a subject of pleasure just is instantiating that material system, no phenomenal ectoplasm required.
Now, I’m not one of those hedonistic utilitarians who believes we should fill the universe with hedonium, eliminating all other potential sources of value. But I will make the modest claim that, ceteris paribus, happiness is good. If we have the capacity to make lots of it cheaply, without major sacrifices to other value-sources, then we should do it! Certain forms of hedonium, given the limited range of experiences we wish to instantiate, could be very simple, and there’s no reason to assume complexity is necessary for basic value.
So I think building hedonium should be an ongoing project for illusionist philosophers and empirical researchers. In §1, I offer a basic formulation of illusionism and explain why I think it’s a justifiable framework for the project. In §2, I describe the implications of illusionism for consciousness research. In §3, I discuss ideas for what the hedonium project would concretely look like.
1. What does illusionism mean?
I think illusionism is ~60% likely to be true, mostly due to being unsatisfied with alternative theories of consciousness. I’ve written quite a bit about this, but others have written even better:
I won’t attempt to argue for illusionism further here. However, I will argue that even if you are not an illusionist, you should still be quite interested in an illusionist-inspired hedonium project:
2. How an illusionist studies consciousness
This section draws heavy inspiration from Keith Frankish, especially here and here. Most of what I am saying here is an attempt to elaborate and precisify those ideas.
The old ideas about consciousness research—that we can “never know” if a physical system is “really” conscious, that nobody has any idea what consciousness is, that consciousness is a binary—all of those make no sense under the illusionist paradigm. Pain & pleasure aren’t correlated with, or even “caused by” physical processes—they are physical processes. So, when you encounter a system, ask not: is it conscious? Or: how are its physical brain and its consciousness linked? Instead ask, purely in functional terms:
…and so on. If your thought is, “okay, great, now we need to figure out which of these things generate consciousness,” then you haven’t taken the illusionist lesson to heart. Imagine you came across a 19th-century biologist who believes in élan vital, asking what is this mysterious essence which makes some things really alive and others, merely dead & mechanical matter. You explain that life is actually just a family of capacities for self-organization, reproduction, homeostasis, and so on. “Ah,” says the biologist, “now which of these things generate life, and how do they do it?” You tell the biologist that these things don’t “generate” life, they are what it means to be alive. “What!” says the biologist, outraged. “You deny the existence of life?!” It’s not just that the 19th-century biologist is incorrect, it’s that they aren’t even working with the right concepts. They think they are looking for life, but they are really looking for élan vital, which they will never find—not least because it doesn’t exist! Meanwhile life is all around them, but they don’t recognize it as such.[1]
So, in trying to build pleasure, an illusionist wants to know: what is typically going on in the brain when we introspectively judge that we are in pleasure? What kind of representations are in play? What kind of dispositions and sensitivities are involved? Whenever we describe some aspect of what pleasure is like—what underlying structure leads us to think pleasure is like that and not like this? These are all difficult empirical questions, but not unknowable in any sense.
Then, there are further philosophical questions: what aspects of the states we call pleasure and pain are the morally relevant ones? For instance, I have a disposition to curse loudly when I experience pain, and laugh when I experience pleasure. But many animals neither speak nor laugh. Does that mean that their pleasure and pain are morally irrelevant? The illusionist cannot simply pass the buck to psychophysical laws: “whatever physical processes generate the private, intrinsic, ineffable qualia of pleasure and pain is what’s relevant.” Instead, illusionists have to work out what features about pleasure & pain that is responsible for our evaluative stance towards them: what is it about pleasure, in purely material terms, which causes us to judge that it is good? What is a ‘judgment’ here, and what does it require? How do these judgments give us reason to act? This will require a careful act of integrating empirical facts into a system of normative value. I have expressed some doubts about whether this is possible, but it seems that outside of a few papers by François Kammerer, it has not been given a serious try.
3. What the hedonium project would look like
Perhaps the strongest argument against starting a hedonium project right now is that it sounds like it would take a lot of detailed neuroscience to pull off, and it seems like neuroscience is way behind where it would need to be to explain much psychology. Mechanistic interpretability is really hard. Neuroscientists have to do interpretability on massively more complicated AGI architectures without white-box access. If this is an empirically tractable project at all, wouldn’t it be better to wait until after the intelligence explosion, when our scientific tools will be much better?
Unfortunately, I think we do need to get cracking on detailed mechanisms for valenced experience before then, even if we can only provide probabilistic guesses based on current science:
Of course, you could argue that in this case we should just work on alignment, stopping entrenchment, and metaphysically agnostic ethical systems. I agree that we should be working on all those approaches, as well as working on the mechanistic, illusionist approach. It seems like there’s a dearth of concrete, shovel-ready projects in digital welfare, and that nobody has given this angle a serious try.
So, keeping in mind the target of helpfully informing near-term alignment and model welfare issues, what is it that such a project should be doing?
I don’t think this would be a substantially expensive project; I could see a small team of dedicated researchers making significant progress. If it’s true that EA is about to see a lot of new funding for ambitious projects, I would feel pretty excited about looking for funding, mentorship, & org support and trying to get this off the ground.
Phenomenal realists are well-aware of this analogy and have offered arguments about why they think consciousness isn’t like élan vital. I am not trying to strawman anyone here; I am trying to articulate how phenomenal realist research proposals look to the illusionist—why they would seem fundamentally mistaken.