[ Question ]

What specifically is the computation -> qualia theory?

by lc3 min read2nd Nov 202122 comments

0

Frontpage

Sometimes people have an argument against something, and they say "I don't understand people's position on x", but they really do understand, it's just not a charitable interpretation. This is not that thing. I do not understand the position that GPT-N can suffer in the same way that an actual physical person or chicken can. It's easier for me to sympathize with the position that the earth is flat or that tarot cards work, because those are things for which direct contradicting evidence doesn't appear to people every day. 

Some of this genuine puzzlement will nevertheless be mixed with rhetorical questions designed to soothe my frustration, but the confusion is real. It's possible that most of this post will come across as belittling, and I apologize - there are clearly people way smarter and more rational than me that believe this. I'm just walking through my entire thought process because to me it's such a strange idea that I don't know which part is actually contested.

Anesthesiology is the branch of medicine dedicated to preventing people from suffering during medical operations. Most people in first world countries have taken anesthetics at some point in their lives. The acceptance or relevance of anesthetics by patients has no simple requirement on intelligence or rationality. It does not depend at all on whether or not someone thinks it's "right" to feel pain in an abstract sense. A person with the intellectual capacity of a chimp and with very limited self awareness can be given anesthetics, and feel pain. Certainly dead people don't feel pain, and the brain is a large part of the process by which pain asserts itself, but dentists do not hand out IQ tests to people to see if they need anesthetics before cutting someone's mouth open. People can even feel pain while they are asleep. The capability of the party in question to "compute" and "simulate" are definitively not requirements for suffering to happen. Any proposal that sentience is the key defining factor in whether or not something can experience things needs to explain why people's emotions and disposition are so easily affected by chemical injections that don't appear to involve or demand any part of their self awareness. 

Yet these very smart, very rational people seem to suspect that if you have a computer that symbolically approximates the thought process of a human, you have a literal human with feelings and moral weight. I'm not even talking about simulating the entire human brain or body, even though I think the idea of 'simulations' suffering is patently absurd. I'm talking about using an AI trained to predict the next word, randomly selecting from the top 80% probability space of those words, and then repeating the loop. Eliezer apparently believes that if there were a pretrained transformer smart enough to accurately predict human dialog, and you ran top-k with a prompt about the holocaust, somehow (I think) the process of text generation would cause suffering. It wouldn't have to be simulating a central nervous system or anything like that. As long as the text was describing human suffering, and an AI computed correct "enough" (enough being unspecified, of course) token probabilities, that's all you need.

What the hell is he talking about? I know this is more than just him because I hear this kind of talk all of the time. I have the same reaction to these ideas that I have to the people who develop really fancy theories about laughter, that have obvious contradictions in 90% of laughter that occurs in daily life. Have these people never heard of drugs? When I take cocaine, the part of my brain that solves math equations is nowhere to be found in the ensuing acute high. It's not an instrumentally relevant component of my body, just like my ability to walk or see.

What is the difference under this theory of "computation therefore qualia" between GPT-N and a human true crime author imagining what it be like to experience a fictional dismemberment? When does my murder novel, or perhaps murder novel writing, become a real example of torture and death under this theory of human experience? Is the difference between the author and GPT-N granularity? Accuracy? If I spend years doing the computations on pen and paper, does that work too? What if I get GPT-N to start writing murder novels and simulate the thought process of GPT-(N-1), a murder victim who features in the story? How would you administer heroin to a pretrained transformer?

0

New Answer
Ask Related Question
New Comment

4 Answers

I don't know whether chickens or GPT-3 feel pain, but I have higher weightings on chickens feeling pain than GPT-3 doing so. However, my reason for having weights in this direction is nothing like yours.

You seem to be reading the term "computation" as being explicit, symbolic computation. It isn't. Even an unconscious human's brain does an enormous amount of computation in the sense being meant, all the time. The idea of "computation" here is an abstraction that refers to the process of changing from one brain state to another, in the sense of the operation of all the neurons, synapses, hormones, neurotransmitters, and anything else that describes how the brain does what it does.

When I take cocaine, the part of my brain that solves math equations is nowhere to be found in the ensuing acute high.

Of course not. The type of computation being discussed is nothing to do with any part of your brain that solves math equations. The behaviour of "solving math equations" is an emulation of an emulation of very different (and much less complicated) computational processes and isn't relevant here at all.

Cocaine does affect the behaviour of your brain, in a way that changes the computational processes being done there in many ways that we can already identify and many that we can't yet. One day maybe we will identify all the ways it affects brain function. It is the effect on those computational processes that matters for this purpose.

More advanced people could, in principle, replace part of someone's brain (maybe to work around brain damage) with a computer-controlled device that emulates exactly the same processes as the original brain area, connects to the same neurons, responds to all the chemical signals in the same way and so on. From the outside, the behaviour could be indistinguishable. What would be the subjective experience of such a person, if even the rest of their own brain doesn't experience any difference?

One hypothesis is that there is no subjective difference. It is very difficult to see how there could be any, since there can be no difference in their responses to questions like "do you feel any different?"

This leads to the thought that maybe this would still apply if you replaced the whole brain with an equally well matched computational device. That would mean that the computational process itself must be experiencing things in the same way that we do.

That in turn raises the question of how different the processes can be from the behaviour of a human brain and still be conscious of experiences in a way that is ethically relevant.

Is a chicken sufficiently similar that we should expect it to have subjective experiences that are morally relevant? Is GPT-3?

Certainly GPT-3 has more verbal capability to express experience concepts in human language, but chickens match more closely in nonverbal respects to how humans behave when we have various basic experiences. I think the latter is better evidence, but not enough to be certain.

You seem to be reading the term "computation" as being explicit, symbolic computation. It isn't. Even an unconscious human's brain does an enormous amount of computation in the sense being meant, all the time. The idea of "computation" here is an abstraction that refers to the process of changing from one brain state to another, in the sense of the operation of all the neurons, synapses, hormones, neurotransmitters, and anything else that describes how the brain does what it does.

I'm going to write a motte and bailey post about this, because this is an abs... (read more)

1JBlack1moIt's certainly not intended as motte-and-bailey, as this model is the one I've held for some time, and as far as I can tell is shared by many other people who consider the topic of machine qualia. I don't know which specific inferences by EY you are referring to, so make sure to include them in your post. Certainly, and I'm saying that brain computation is not the same thing as general intelligence. Very few people argue that general intelligence is required for experiencing qualia, and even those that do argue do not equate brain computations only with general intelligence as you appear to be doing. In this model, brain computation is required for general intelligence, but the type of brain computation being discussed occurs even in ants.
1JBlack1moOf course it's possible to lie. The argument goes the opposite direction: if in such a scenario the person really did feel different, it would be impossible for them to tell the truth, and that seems an extraordinary claim.
1lc13dThat makes more sense to me, actually.

After some more reading I think I understand better the (IMO completely bonkers) logic behind worrying if a GPT-N system is suffering. There are three pillars to this idea:


1. Suffering is the result of computation. Specifically, it's caused by computing one of a class of algorithms, assumed to be vaguely close to the ones that humans use to make lots of generally intelligent predictions. The form or manner in which computation happens is thus unimportant; whether it's human brains or computers, the hardware is an implementation detail. A person with rocks to move around can cause untold suffering, as long as those rocks "mean" states of a universe. In xkcd's case, a simulation of physics in which some atoms are running one of The Algorithms, like a matryoshka doll. 

What The Algorithms are, how one might be run a variation to produce arbitrary amounts of happiness, etc., by the open admission of the S.I.C. people, is unclear. Sometimes SIC people imply that suffering is the result of an agent having a mismatch between a utility function, their observations, and their environment. But humans don't have utility functions - they experience emotions for a plethora of very circumstantial reasons, not necessarily in reaction to any consistent values whatsoever. We have an array of qualitatively dissilimar emotions - the pleasure/pain thing is just a shorthand. Often the sources of these emotions have very physiologically tractable sources, and people's responses to stimuli manipulated or changed by doctors. There's lots of psychological evidence that says that humans' decision making process is nearly independent of their happiness and that humans don't actually try all that hard to make themselves more happy. Whatever the Algorithm is, it seems like SIC campers are really ready to jump out into the motte and start talking about intelligence like it's some obvious source of qualia when it's clearly much more complicated.

It's also unclear to me if SIC campers think that accidental rock movement, like with a hypothetical boltzmann brain, can also suffer if it starts to resemble a 'mind' enough from some interpretation. I'm assuming we're all atheists here, so clearly the rocks don't have to be "mean" anything to another entity, or else human's couldn't have qualia either. What is the "correct" way to identify if something is running an algorithm or is just engaging in simple random movements? It seems to me like there should be an infinite number of ways to interpret atoms' vibrations as having information or even being transformed in a way that approximates the operation of an algorithm. I don't know if any SIC campers actually worry about this, if they have specific requirements on how to tell if atoms are running algorithms, how likely they think this is to happen in practice, etc.

2. Conveniently, only the algorithms that powerful humans can run are capable of inducing qualia. Chickens do not suffer because they are not capable of "self-awareness", the feature that just so happens to be what led humans to take over the world. This is despite the fact that human suffering seems to happen independently of the correct operation of any of the cool rational reasoning skills that make humans powerful, and that suffering is instead conditional entirely on very boring biocomponents like working central nervous systems, and bloodstream contents. 

I'm almost confident enough to call this idea fantastically and obviously wrong. This seems pretty much perfectly analogous to the arguments that colonial slavemasters made about how slaves were just so much less sophisticated and intelligent, and so felt less pain, so don't worry about all of the hard labor. Ignore the way  animals signal pain exactly like regular people do. They don't have the spark of life self-awareness.

3. Given these premises, in the course of its operation, a GPT-N system has to sometimes predict what humans do. If the algorithm is "close enough" to a perfect simulation of human death, it might be causing actual human like suffering via latent and unspecified side effects of computation. The underlying operations don't have to be anything even analogous to the human brain - you can get here by multiplying enough matrices if you want - as long as the model is predicting human behavior with enough granularity and accuracy. Presumably humans could do this too if they had enough time, paper, and abacuses, as per #1.

It seems to me like there should be an infinite number of ways to interpret atoms’ vibrations as having information or even being transformed in a way that approximates the operation of an algorithm. I don’t know if any SIC campers actually worry about this, if they have specific requirements on how to tell if atoms are running algorithms, how likely they think this is to happen in practice, etc.

People do indeed worry about this, leading to things like 'Solomonoff-weighted utilitarianism' that assign higher moral relevance to minds with short description lengths.

The only core idea here is (1).

(2) is completely superfluous. Discard it.

(3) is a possible reason for concern, not any sort of "this is definitely happening" argument.

Given that we don't know what processes have qualia, that humans definitely do have qualia (well, at least I do), there's at least some tiny amount more evidence that something simulating a human really well experiences qualia than things that don't.

Any proposal that sentience is the key defining factor in whether or not something can experience things needs to explain why people's emotions and disposition are so easily affected by chemical injections that don't appear to involve or demand any part of their self awareness. 

 

Presumably such a explanation would look like this:  

Pain happens when your brain predicts that bad things are going to happen to it in the future.  Morphine interferes with the body's ability to make such predictions therefore it decreases the ability to feel pain.

One  pleasurable  feeling is the ability to  correct predict input stimuli.  Cocaine causes the brain to experience a single overwhelming stimuli which is easy to predict and therefore pleasurable.

Note:  I am not saying either of these theories are correct (I don't known  nearly enough neurobiology).  But these  are the types of theories someone who thinks that qualia (pleasure/pain) are purely computational would  assert.  Namely, chemicals alter our qualia by altering the way that we compute different types of information.

There's a difference between saying that pleasure/pain accompany (mis)prediction , and saying that they are identical. The former doesn't guarantee anything about an AI .

I think you're confused by what the word "computation" means, in this context.

If you take a fully-detailed mathematical description of physics, and combine it with a fully-detailed description of the configuration of atoms that makes up a brain, and you play the simulation forward, that's a computation. It's also a mind. You can abstract this a little bit; eg, you might only specify neurons and their connections, rather than individual atoms. That's still mostly the same thing, it's still a computation, and it's still a mind. And if the neuron-connections you specify are the neuron-connections of a person who's asleep, that, too, is a mind, and is a computation.

All of these computations involve really, really astronomically large numbers of elements and steps. Whether the world's largest supercomputers are big enough to run this sort of simulation, appropriately abstracted, is an open question.

Whereas when you say:

I assure you all that when I take cocaine, the feature of my brain that solves math equations is nowhere to be found in the ensuing acute high.

You have jumped to talking about something else. Solving math equations is a thing that some minds can do. But these equations are not minds, because they're too simple and too small to contain all of the necessary features.

What is the difference between GPT-N doing it and a true crime author imagining what it be like to experience a fictional dismemberment? When does my murder novel become a real example of torture and death to these people? Is the difference granularity? Accuracy? If I spend years doing the computations on pen and paper, does that work too? 

It's a combination of having certain computational features, and being a large-enough computation for it to be possible to contain those features. GPT-3 does not have those features, but GPT-N potentially could, if the algorithm details are filled in in certain ways. Nothing done by humans on pen and paper is enough; it falls short by quite a lot of orders of magnitude.

Relevant reading: Thou Art Physics, How an Algorithm Feels from the Inside

I think you are confusing necessity and sufficiency. Of course it takes an enormous amount of compute to perform a fine grained simulation of physics...so it is necessary to perform a simulation of physics in order to achieve artificial qualia. But it's still only a simulation: if no simulation, however detailed, is sufficient to have real qualia, it doesn't .

When I said:

I assure you all that when I take cocaine, the feature of my brain that solves math equations is nowhere to be found in the ensuing acute high.

I meant 'part', not feature. I was talking about the general capability of my brain for broad inference, not the act of solving equations. My point was that the parts of me that allow me to solve equations aren't functionally relevant to the experience. Eliezer points out how a chicken and GPT-3 have the same inference capabilities as if people that disagree with him on the chicken front are disagreeing about where the inference cutoff lies, and not about which biocomponent is causing pain.

If you take a fully-detailed mathematical description of physics, and combine it with a fully-detailed description of the configuration of atoms that makes up a brain, and you play the simulation forward, that's a computation. It's also a mind. You can abstract this a little bit; eg, you might only specify neurons and their connections, rather than individual atoms. That's still mostly the same thing, it's still a computation, and it's still a mind. And if the neuron-connections you specify are the neuron-connections of a person who's asleep, that, too, is

... (read more)

It's a combination of having certain computational features, and being a large-enough computation for it to be possible to contain those features. GPT-3 does not have those features, but GPT-N potentially could, if the algorithm details are filled in in certain ways. Nothing done by humans on pen and paper is enough; it falls short by quite a lot of orders of magnitude.

This is handwaving away the argument. Suppose someone comes up with a much smaller version of GPT-3 that is 'n' orders of magnitude more effective at predicting text tomorrow. A hundred thou... (read more)

1JBlack1moI'm moderately sure that GPT-3 is not sentient, but I accept the possibility that a hypothetical society with enough pen-and-paper might well be able to create something that would reasonably be called a person, and that the person so created could experience horrible things. I'm virtually certain that a mere hundred thousand people working for ten years won't do it, though.
1lc1moI don't know if I'm reading into this too much, but it's almost like even when phrasing this, you have to obfuscate it because it's just too ridiculous. To be clear, you believe that actual pen and paper and the computations together would be a person.
1JBlack1moIt's not obfuscated, it's just being precise. I believe that quantity matters, and you're trying to handwave away the quantities involved to make it seem ridiculous. In this case, the quantities required are so large that I'm not even sure that a society could coordinate such an immense effort. But yes, setting complexity and scale difficulties aside, I do believe that. However, that handwaving is comparable to dismissing all life on Earth as just "chemicals mixed together". It has connotations like maybe as many as 10 different molecules instead of billions, a featureless soup instead of very detailed structures, thermodynamic equilibrium instead of active homeostasis far from equilibrium. In other words, a dismissive rhetorical trick instead of actually exploring the hypothesis.
1lc13dThe reason it seems silly to me has nothing to do with the quantity involved and everything to do with how abstract the suffering seems, and how ill-defined the map between ink and human experience is.
1JBlack13dIt's only abstract because you're choosing to visualize at the wrong level. Human suffering would seem equally abstract if you were incapable of seeing a person's behaviour in real time, and could only examine them on the scale of a single cell at a time, and slowed down a billion to one.
2 comments, sorted by Highlighting new comments since Today at 7:18 AM

It feels like you give several closely related arguments, each individually appearing to be against something, but after reading this I'm still not clear on what you're arguing against or even if there's a unified thing here rather than several different things.

Maybe it would help me if you were to take a step back, cite some sources that you specifically want to understand and/or argue against, and explain your thoughts in a way that looks more like a brief history than a long argument?

"Why is he making so many replies?"

https://www.lesswrong.com/posts/fBF8GuBmQnNCBzh6Y/multiple-arguments-multiple-comments