GAZP vs. GLUT

Followup toThe Generalized Anti-Zombie Principle

In "The Unimagined Preposterousness of Zombies", Daniel Dennett says:

To date, several philosophers have told me that they plan to accept my challenge to offer a non-question-begging defense of zombies, but the only one I have seen so far involves postulating a "logically possible" but fantastic being — a descendent of Ned Block's Giant Lookup Table fantasy...

A Giant Lookup Table, in programmer's parlance, is when you implement a function as a giant table of inputs and outputs, usually to save on runtime computation.  If my program needs to know the multiplicative product of two inputs between 1 and 100, I can write a multiplication algorithm that computes each time the function is called, or I can precompute a Giant Lookup Table with 10,000 entries and two indices.  There are times when you do want to do this, though not for multiplication—times when you're going to reuse the function a lot and it doesn't have many possible inputs; or when clock cycles are cheap while you're initializing, but very expensive while executing.

Giant Lookup Tables get very large, very fast.  A GLUT of all possible twenty-ply conversations with ten words per remark, using only 850-word Basic English, would require 7.6 * 10585 entries.

Replacing a human brain with a Giant Lookup Table of all possible sense inputs and motor outputs (relative to some fine-grained digitization scheme) would require an unreasonably large amount of memory storage.  But "in principle", as philosophers are fond of saying, it could be done.

The GLUT is not a zombie in the classic sense, because it is microphysically dissimilar to a human.  (In fact, a GLUT can't really run on the same physics as a human; it's too large to fit in our universe.  For philosophical purposes, we shall ignore this and suppose a supply of unlimited memory storage.)

But is the GLUT a zombie at all?  That is, does it behave exactly like a human without being conscious?

The GLUT-ed body's tongue talks about consciousness.  Its fingers write philosophy papers.  In every way, so long as you don't peer inside the skull, the GLUT seems just like a human... which certainly seems like a valid example of a zombie: it behaves just like a human, but there's no one home.

Unless the GLUT is conscious, in which case it wouldn't be a valid example.

I can't recall ever seeing anyone claim that a GLUT is conscious.  (Admittedly my reading in this area is not up to professional grade; feel free to correct me.)  Even people who are accused of being (gasp!) functionalists don't claim that GLUTs can be conscious.

GLUTs are the reductio ad absurdum to anyone who suggests that consciousness is simply an input-output pattern, thereby disposing of all troublesome worries about what goes on inside.

So what does the Generalized Anti-Zombie Principle (GAZP) say about the Giant Lookup Table (GLUT)?

At first glance, it would seem that a GLUT is the very archetype of a Zombie Master—a distinct, additional, detectable, non-conscious system that animates a zombie and makes it talk about consciousness for different reasons.

In the interior of the GLUT, there's merely a very simple computer program that looks up inputs and retrieves outputs.  Even talking about a "simple computer program" is overshooting the mark, in a case like this.  A GLUT is more like ROM than a CPU.  We could equally well talk about a series of switched tracks by which some balls roll out of a previously stored stack and into a trough—period; that's all the GLUT does.

A spokesperson from People for the Ethical Treatment of Zombies replies:  "Oh, that's what all the anti-mechanists say, isn't it?  That when you look in the brain, you just find a bunch of neurotransmitters opening ion channels?  If ion channels can be conscious, why not levers and balls rolling into bins?"

"The problem isn't the levers," replies the functionalist, "the problem is that a GLUT has the wrong pattern of levers.  You need levers that implement things like, say, formation of beliefs about beliefs, or self-modeling...  Heck, you need the ability to write things to memory just so that time can pass for the computation.  Unless you think it's possible to program a conscious being in Haskell."

"I don't know about that," says the PETZ spokesperson, "all I know is that this so-called zombie writes philosophical papers about consciousness.  Where do these philosophy papers come from, if not from consciousness?"

Good question!  Let us ponder it deeply.

There's a game in physics called Follow-The-Energy.  Richard Feynman's father played it with young Richard:

    It was the kind of thing my father would have talked about:  "What makes it go?  Everything goes because the sun is shining."   And then we would have fun discussing it:
    "No, the toy goes because the spring is wound up," I would say.  "How did the spring get wound up?" he would ask.
    "I wound it up."
    "And how did you get moving?"
    "From eating."
    "And food grows only because the sun is shining.   So it's because the sun is shining that all these things are moving."   That would get the concept across that motion is simply the transformation of the sun's power.

When you get a little older, you learn that energy is conserved, never created or destroyed, so the notion of using up energy doesn't make much sense.  You can never change the total amount of energy, so in what sense are you using it?

So when physicists grow up, they learn to play a new game called Follow-The-Negentropy—which is really the same game they were playing all along; only the rules are mathier, the game is more useful, and the principles are harder to wrap your mind around conceptually.

Rationalists learn a game called Follow-The-Improbability, the grownup version of "How Do You Know?"  The rule of the rationalist's game is that every improbable-seeming belief needs an equivalent amount of evidence to justify it.  (This game has amazingly similar rules to Follow-The-Negentropy.)

Whenever someone violates the rules of the rationalist's game, you can find a place in their argument where a quantity of improbability appears from nowhere; and this is as much a sign of a problem as, oh, say, an ingenious design of linked wheels and gears that keeps itself running forever.

The one comes to you and says:  "I believe with firm and abiding faith that there's an object in the asteroid belt, one foot across and composed entirely of chocolate cake; you can't prove that this is impossible."  But, unless the one had access to some kind of evidence for this belief, it would be highly improbable for a correct belief to form spontaneously.  So either the one can point to evidence, or the belief won't turn out to be true.  "But you can't prove it's impossible for my mind to spontaneously generate a belief that happens to be correct!"  No, but that kind of spontaneous generation is highly improbable, just like, oh, say, an egg unscrambling itself.

In Follow-The-Improbability, it's highly suspicious to even talk about a specific hypothesis without having had enough evidence to narrow down the space of possible hypotheses.  Why aren't you giving equal air time to a decillion other equally plausible hypotheses?  You need sufficient evidence to find the "chocolate cake in the asteroid belt" hypothesis in the hypothesis space—otherwise there's no reason to give it more air time than a trillion other candidates like "There's a wooden dresser in the asteroid belt" or "The Flying Spaghetti Monster threw up on my sneakers."

In Follow-The-Improbability, you are not allowed to pull out big complicated specific hypotheses from thin air without already having a corresponding amount of evidence; because it's not realistic to suppose that you could spontaneously start discussing the true hypothesis by pure coincidence.

A philosopher says, "This zombie's skull contains a Giant Lookup Table of all the inputs and outputs for some human's brain."  This is a very large improbability.  So you ask, "How did this improbable event occur?  Where did the GLUT come from?"

Now this is not standard philosophical procedure for thought experiments.  In standard philosophical procedure, you are allowed to postulate things like "Suppose you were riding a beam of light..." without worrying about physical possibility, let alone mere improbability.  But in this case, the origin of the GLUT matters; and that's why it's important to understand the motivating question, "Where did the improbability come from?"

The obvious answer is that you took a computational specification of a human brain, and used that to precompute the Giant Lookup Table.  (Thereby creating uncounted googols of human beings, some of them in extreme pain, the supermajority gone quite mad in a universe of chaos where inputs bear no relation to outputs.  But damn the ethics, this is for philosophy.)

In this case, the GLUT is writing papers about consciousness because of a conscious algorithm.  The GLUT is no more a zombie, than a cellphone is a zombie because it can talk about consciousness while being just a small consumer electronic device.  The cellphone is just transmitting philosophy speeches from whoever happens to be on the other end of the line.  A GLUT generated from an originally human brain-specification is doing the same thing.

"All right," says the philosopher, "the GLUT was generated randomly, and just happens to have the same input-output relations as some reference human."

How, exactly, did you randomly generate the GLUT?

"We used a true randomness source—a quantum device."

But a quantum device just implements the Branch Both Ways instruction; when you generate a bit from a quantum randomness source, the deterministic result is that one set of universe-branches (locally connected amplitude clouds) see 1, and another set of universes see 0.  Do it 4 times, create 16 (sets of) universes.

So, really, this is like saying that you got the GLUT by writing down all possible GLUT-sized sequences of 0s and 1s, in a really damn huge bin of lookup tables; and then reaching into the bin, and somehow pulling out a GLUT that happened to correspond to a human brain-specification.  Where did the improbability come from?

Because if this wasn't just a coincidence—if you had some reach-into-the-bin function that pulled out a human-corresponding GLUT by design, not just chance—then that reach-into-the-bin function is probably conscious, and so the GLUT is again a cellphone, not a zombie.  It's connected to a human at two removes, instead of one, but it's still a cellphone!  Nice try at concealing the source of the improbability there!

Now behold where Follow-The-Improbability has taken us: where is the source of this body's tongue talking about an inner listener?  The consciousness isn't in the lookup table.  The consciousness isn't in the factory that manufactures lots of possible lookup tables.  The consciousness was in whatever pointed to one particular already-manufactured lookup table, and said, "Use that one!"

You can see why I introduced the game of Follow-The-Improbability.  Ordinarily, when we're talking to a person, we tend to think that whatever is inside the skull, must be "where the consciousness is".  It's only by playing Follow-The-Improbability that we can realize that the real source of the conversation we're having, is that-which-is-responsible-for the improbability of the conversation—however distant in time or space, as the Sun moves a wind-up toy.

"No, no!" says the philosopher.  "In the thought experiment, they aren't randomly generating lots of GLUTs, and then using a conscious algorithm to pick out one GLUT that seems humanlike! I am specifying that, in this thought experiment,  they reach into the inconceivably vast GLUT bin, and by pure chance pull out a GLUT that is identical to a human brain's inputs and outputs!  There!  I've got you cornered now!  You can't play Follow-The-Improbability any further!"

Oh.  So your specification is the source of the improbability here.

When we play Follow-The-Improbability again, we end up outside the thought experiment, looking at the philosopher.

That which points to the one GLUT that talks about consciousness, out of all the vast space of possibilities, is now... the conscious person asking us to imagine this whole scenario.  And our own brains, which will fill in the blank when we imagine, "What will this GLUT say in response to 'Talk about your inner listener'?"

The moral of this story is that when you follow back discourse about "consciousness", you generally find consciousness.  It's not always right in front of you.  Sometimes it's very cleverly hidden.  But it's there.  Hence the Generalized Anti-Zombie Principle.

If there is a Zombie Master in the form of a chatbot that processes and remixes amateur human discourse about "consciousness", the humans who generated the original text corpus are conscious.

If someday you come to understand consciousness, and look back, and see that there's a program you can write which will output confused philosophical discourse that sounds an awful lot like humans without itself being conscious—then when I ask "How did this program come to sound similar to humans?" the answer is that you wrote it to sound similar to conscious humans, rather than choosing on the criterion of similarity to something else.  This doesn't mean your little Zombie Master is conscious—but it does mean I can find consciousness somewhere in the universe by tracing back the chain of causality, which means we're not entirely in the Zombie World.

But suppose someone actually did reach into a GLUT-bin and by genuinely pure chance pulled out a GLUT that wrote philosophy papers?

Well, then it wouldn't be conscious.  IMHO.

I mean, there's got to be more to it than inputs and outputs.

Otherwise even a GLUT would be conscious, right?


Oh, and for those of you wondering how this sort of thing relates to my day job...

In this line of business you meet an awful lot of people who think that an arbitrarily generated powerful AI will be "moral".  They can't agree among themselves on why, or what they mean by the word "moral"; but they all agree that doing Friendly AI theory is unnecessary.  And when you ask them how an arbitrarily generated AI ends up with moral outputs, they proffer elaborate rationalizations aimed at AIs of that which they deem "moral"; and there are all sorts of problems with this, but the number one problem is, "Are you sure the AI would follow the same line of thought you invented to argue human morals, when, unlike you, the AI doesn't start out knowing what you want it to rationalize?"  You could call the counter-principle Follow-The-Decision-Information, or something along those lines.  You can account for an AI that does improbably nice things by telling me how you chose the AI's design from a huge space of possibilities, but otherwise the improbability is being pulled out of nowhere—though more and more heavily disguised, as rationalized premises are rationalized in turn.

So I've already done a whole series of posts which I myself generated using Follow-The-Improbability.  But I didn't spell out the rules explicitly at that time, because I hadn't done the thermodynamic posts yet...

Just thought I'd mention that.  It's amazing how many of my Overcoming Bias posts would coincidentally turn out to include ideas surprisingly relevant to discussion of Friendly AI theory... if you believe in coincidence.

 

Part of the Zombies subsequence of Reductionism

Next post: "Belief in the Implied Invisible"

Previous post: "The Generalized Anti-Zombie Principle"

166 comments, sorted by
magical algorithm
Highlighting new comments since Today at 4:03 PM
Select new highlight date
Moderation Guidelines: Reign of Terror - I delete anything I judge to be annoying or counterproductiveexpand_more

Internal state is not necessary. Consider a function f mapping strings to strings by means of a lookup table. Here are some examples of f evaluated with well-chosen inputs:

f("Hi, Dr. S here, how are you now that you're a lookup table?") = "Very well, thank you. I notice no difference."

f("Hi, Dr. S here, how are you now that you're a lookup table? Really, none at all?") = "Yes, really no differences at all."

f("Hi, Dr. S here, how are you now that you're a lookup table? You have insulted my entire family!") = "I know you well enough to know that my last reply could not possibly have insulted you; someone must be feeding me fake input histories again."

There should probably be timestamps in the input histories but that's an implementation detail. For what it's worth, I hold that f is conscious.

"Otherwise even a GLUT would be conscious, right?"

I have to admit that this sounds crazy, and that I don't really understand what's going on. But it looks like it's logically necessary that lookup tables can be conscious. As far as we know, the Universe, and everything in it, can be simulated on a giant Turing machine. What is a Turing machine, if not a lookup table? Granted, most Turing machines use a much smaller set of symbols than a GLUT- base 5 or base 10 instead of base 10^10^50- but how would that change a system from being "non-conscious" to being "conscious"? And while a Turing machine has a state register, this can be simulated by just using N lookup tables instead of one lookup table. It seems like we have to believe that 1), the mathematical structure of a UTM relative to a giant lookup table, which is very minimal indeed, is the key element required for consciousness, or 2), the Universe is not Turing-computable, or 3), consciousness does not exist.

I have to admit that this sounds crazy, and that I don't really understand what's going on. But it looks like it's logically necessary that lookup tables can be conscious. As far as we know, the Universe, and everything in it, can be simulated on a giant Turing machine. What is a Turing machine, if not a lookup table?

But it is not logically necessary that such a universe would contain genuine consciousness. It might be a zombie universe.

1), the mathematical structure of a UTM relative to a giant lookup table, which is very minimal indeed, is the key element required for consciousness, or 2), the Universe is not Turing-computable, or 3), consciousness does not exist.

or (4) the universe is not simulated or (5) consciousness is not simulable, evne if physics is.

PK, Phil Goetz, and Larry D'Anna are making a crucial point here but I'm afraid it is somewhat getting lost in the noise. The point is (in my words) that lookup tables are a philosophical red herring. To emulate a human being they can't just map external inputs to external outputs. They also have to map a big internal state to the next version of that big external state. (That's what Larry's equations mean.)

If there was no internal state like this, a GLUT couldn't emulate a person with any memory at all. But by hypothesis, it does emulate a person (perfectly). So it must have this internal state.

And given that a GLUT is maintaining a big internal state it is equivalent to a Turing machine, as Phil says.

But that means that is can implement any computationally well defined process. If we believe that consciousness can be a property of some computation then GLUTs can have consciousness. This isn't even a stretch, it is totally unavoidable.

The whole reason that philosopher talk about GLUTs, or that Searle talks about the Chinese room, is to try to trick the reader into being overwhelmed by the intuition that "that can't possibly be conscious" and to STOP THINKING.

Looking at this discussion, to some extent that works! Most people didn't say "Hmmm, I wonder how a GLUT could emulate a human..." and then realize it would need internal state, and the internal state would be supporting a complex computational process, and that the GLUT would in effect be a virtual machine, etc.

This is like an argument where someone tries to throw up examples that are so scary, or disgusting, or tear jerking, or whatever that we STOP THINKING and vote for whatever they are trying to sneak through. In other words it does not deserve the honor of being called an argument.

This leaves the very interesting question of whether a computational process can support consciousness. I think yes, but the discussion is richer. GLUTs are a red herring and don't lead much of anywhere.

I mean, there's got to be more to it than inputs and outputs.

Otherwise even a GLUT would be conscious, right?

Eliezer, I suspect you are not being 100% honest here. I don't have any problems with a GLUT being conscious.

If the GLUT is conscious, then there is consciousness in the Zombie World. There cannot be consciousness in the Zombie World, therefore the GLUT cannot be conscious. If the GLUT is not conscious, something else must be looking up the inputs in the GLUT, and that thing must be conscious. Rinse - Repeat.

This is why the epiphenomenal argument is logically impossible - either the Zombie World is not exactly the same as ours (precluded by the framing of the thought experiment) or there is consciousness in the Zombie World (also precluded by the framing of the thought experiment). They are mutually exclusive. A Zombie Master with a GLUT does not solve the problem for the epiphenomenal position - it's just Zombie Masters and GLUTs all the way down.

If the GLUT is not conscious, something else must be looking up the inputs in the GLUT, and that thing must be conscious.

This is an incorrect synthesis, and likely an incorrect conclusion. Eliezer is saying it is the process by which the table is populated that involves consciousness, not the thing that does the picking.

Right, I was arguing Roland's point, not Eliezer's, and I don't see where I disagreed with Eliezer in any way.

Roland just said the GLUT is conscious, which means by definition it isn't in a Zombie World, because the definition of a Zombie World is one that is apparently identical to ours, minus consciousness.

I'm not sure where I screwed the synthesis up here, Eliezer's post doesn't really come into it except for framing the subject of Zombie Worlds, GLUTs, and consciousness.

I was just saying if the GLUT is conscious then the Anti-Zombie position automatically wins the Zombie World GLUT argument, by definition. Perhaps I should have just said it exactly like that?

Please reread the bit I quoted. I am not trying to be pedantic, and it's possible that either I am misreading you, or that you just didn't write quite what you'd intended. Speaking of the case where the GLUT itself is not conscious, which was the sole of my focus, it seems to me that you said that the thing that is "looking up the inputs in the GLUT" must be conscious. This seems to mean "thing that is performing the lookup operation", which is different than "thing that stored the data to be looked up." Did I misunderstand you?

I didn't misspeak, even though the argument I gave wasn't the exact same one that Eliezer gave it is essentially the exact same argument. The "rinse, repeat" was meant to suggest you keep going with it ad infinitum, and the very next step I came up with was the exact same as Eliezer's. It's a reference to washing directions on shampoo bottles, and I've honestly never had anyone get confused by it, so I apologize.

The point is, if the thing looking up the inputs (to continue where I left off) isn't conscious, then the thing that created the thing that looks up the inputs is probably conscious. If the thing that created the thing that created the GLUT isn't conscious (e.g. a true random code generator that happens to produce the GLUT that matches our universe) then the thing that chose that GLUT out of the multitude of others is probably conscious. This is exactly Eliezer's argument, unless I have completely misunderstood it, and if I have I would love to be corrected (I've only been actively engaged in this kind of thinking for a little over a year now, so I'm still very much a newbie). As it is, I don't see what is different in principal between my point and Eliezer's.

My point was in regards to Roland's argument, which was that he didn't mind the GLUT being consciousness in his Zombie World. I was attempting to point out that if the GLUT is conscious then the anti-zombie principle is automatically validated on the grounds that it's not a Zombie World in that case.

In order for epiphenomenalists to effectively argue the Zombie World using a GLUT, it cannot be conscious itself. Eliezer argued (in a nutshell) that if there was a conscious mind behind the creation of the GLUT, then the GLUT was simply a tool of the conscious mind, and that the GLUT wasn't actually running things, the consciousness behind it was. This is true regardless of where in the process the consciousness is, the point is that it is there somewhere and has a meaningful affect on the universe it exists in. Any world that his this kind of connection to a consciousness by definition can't be a real Zombie World.

If consciousness has no affect on the universe then it is meaningless. This has been my understanding of Eliezer's position throughout this entire series.

"The GLUT is no more a zombie, than a cellphone is a zombie because it can talk about consciousness while being just a small consumer electronic device. The cellphone is just transmitting philosophy speeches from whoever happens to be on the other end of the line. A GLUT generated from an originally human brain-specification is doing the same thing."

You begin by saying that you are using "zombie" in a broader-than-usual sense, to denote something that "behave[s] exactly like a human without being conscious". The GLUT was constructed by observing googols of humans, but no human being plays a part in its operation. Are you going to call it conscious just because humans were an input to the design process? And even that's not true, in the extremely improbable but still possible case where the GLUT is generated by a random process. Is the presence of consciousness supposed to depend on the manner of creation, even though the result be physically identical?

It is possible that the point of this essay was just to say that if something talks with facility about being conscious, then with overwhelming probability the real thing is somewhere causally upstream, and that you were not taking a stand as to whether the GLUT in itself is conscious or not. But the evidence suggests otherwise: you do say that the randomly generated GLUT is not conscious, and you say that the GLUT generated by brute-force observation "is not a zombie". In which case I ask again, Is the presence of consciousness supposed to depend on the manner of creation, even though the result be physically identical?

And a bonus question: Suppose we incrementally modify the GLUT so that more and more of its responses are generated through computation, rather than just being looked up. Evidently there is something of a continuum between pure GLUT and shortest possible program implementing exactly the same responses. Where in this continuum is the boundary between consciousness and unconsciousness?

Unless you think it's possible to program a conscious being in Haskell."

Ahemhem. Haskell is as fine a turing complete language; we just like to have our side effects explicit!

Also, can we just conclude that "consciousness" is the leakiest of surface generalizations ever? If I one day get the cog-psy skills I am going to run a stack-trace on what makes us say "consciousness" without knowing diddy about what it is.

As a budding AI researcher, I am frankly offended by philosophers pretending to be wise like that. No. There is no such thing as "consciousness" because it is not even a bucket to put things in. It's metal shreds. You are a some sort of self-introspective algorithm implemented on a biochemical computing substrate, so let's make the blankness of our maps self-evident by calling it "magic" or something.

Glaring redundancy aside, isn't "self-introspective" just as intensionally valid or void as "conscious"?

Yes, probably. It is a really good idea to taboo any and all of "conscious," "self-," "introspective," "thinking," and so on when doing AI work, or so I heard.

Ahemhem. Haskell is as fine a turing complete language; we just like to have our side effects explicit!

I don't think we even need turing-completeness, really. Take a total language, and give it a loop-timeout of say 3^^^^3. It can compute anything we care about, and is still guaranteed to terminate in bounded time.

(I'm reminded of Godel's letter discussing P=NP - should such an algorithm be found, mathematicians could be replaced by a machine that searches all proofs with less than a few million symbols; anything that couldn't be proved with that many symbols would be of little to no interest to us.)

(I'm reminded of Godel's letter discussing P=NP - should such an algorithm be found, mathematicians could be replaced by a machine that searches all proofs with less than a few million symbols; anything that couldn't be proved with that many symbols would be of little to no interest to us.)

Or, at least, they could be replaced by people who can understand and seek out proofs that are relevant to us out of the arbitrarily large number of useless ones. So mathematicians basically.

Unless for any NP problem there exists an algorithm which solves it in just O(N^300) time.

NP=P is not the same as NP=Small P

Likewise, EXPTIME doesn't mean Large EXPTIME -- an algorithm running in exp(1e-15*N) seconds is asymptotically slower than one running in N^300 seconds, but it is faster for pretty much all practical purposes.

I once read an Usenet post or Web page along the lines of “There are two kinds of numbers: those smaller than Graham's number and those larger than Graham's number. Computational complexity theory traditionally only concerns itself with the latter, but only the former are relevant to real-world problems.”

Complexity theorists don't know anything, but they at least know that it's impossible to solve all NP problems in O(N^300) time. In fact they know it's impossible to solve all P problems in O(N^300) time.

http://en.wikipedia.org/wiki/Time_hierarchy_theorem

I think the charitable interpretation is that Eliezer meant someone might figure out an O(N^300) algorithm for some NP-complete problem. I believe that's consistent with what the complexity theorists know, it certainly implies P=NP, but it doesn't help anyone with the goal of replacing mathematicians with microchips.

I don't think that interpretation is necessary. A better one is that even if all NP problems could be solved in O(N^300) time, we'd still need mathematicians.

Sewing-Machine correctly pointed out, above, that this contradicts what we already know.

Are you saying that the counterfactual implication contradicts what we already know, or that the antecedent of the counterfactual implication contradicts what we already know?

I'd be surprised by the former, and the latter is obvious from that it is a counterfactual.

I'm not really comfortable with counterfactuals, when the counterfactual is a mathematical statement. I think I can picture a universe in which isolated pieces of history or reality are different; I can't picture a universe in which the math is different.

I suppose such a counterfactual makes sense from the standpoint of someone who does not know the antecedent is mathematically impossible, and thinks rather that it is a hypothetical. I was trying to give a hypothetical (rather than a counterfactual) with the same intent, which is not obviously counterfactual given the current state-of-the-art.

When I first read that TDT was about counterfactuals involving logically impossible worlds I was uncomfortable with that but I wasn't sure why, and when I first read about the five-and-ten problem I dismissed it as wordplay, but then it dawned on me that the five-and-ten problem is indeed what you get if you allow counterfactuals to range over logically impossible worlds.

I was having trouble articulating why I was uncomfortable reasoning under a mathematical counterfactual, but more comfortable reasoning under a mathematical hypothesis that might turn out to be false. This comment helped me clarify that for myself.

I'll explain my reasoning using Boolean logic, since it's easier to understand that way, but obviously the same problem must occur with Bayesian logic, since Bayes generalizes Boole.

Suppose we are reasoning about the effects of a mathematical conjecture P, and conclude that P -> Q, and ¬P -> Q'. Let's assume Q and Q' can't both be true, because we're interested in the difference between how the world would look if P were true, and how the world would look if P' were true. Let's assume we don't have any idea which of P or ¬P is true. We can't also have concluded P -> Q', because then the contradiction would allow as to conclude ¬P, and for the same reason we can't also have concluded ¬P -> Q. When we assume P, we only have one causal chain leading us to distinguish between Q and Q', so we have an unambiguous model of how the universe will look under assumption P. This is true even if P turns out to be false, because we are aware of a chain of causal inferences beginning with P leading to only one conclusion.

However, the second we conclude ¬P, we have two contradictory causal chains starting from P: P -> Q, and P -> ¬P -> Q', so our model of a universe where P is true is confused. We can no longer make sense of this counterfactual, because we are no longer sure which causal inferences to draw from the counterfactual.

To elaborate on this a little bit, you can think of the laws of physics as a differential equation, and the universe as a solution. You can imagine what would happen if the universe passed through a different state (just solve the differential equation again, with new initial conditions), or even different physics (solve the new differential equation), but how do you figure out what happens when calculus changes?

anything that couldn't be proved with that many symbols would be of little to no interest to us

Are you sure? Maybe there are exist concise, powerful theorems that have really long proofs.

Oh good grief, since everyone here is intent on nitpicking the observation to death, here is his bloody letter: http://rjlipton.wordpress.com/the-gdel-letter/

It's interesting that Eliezer never heard anyone say that a GLUT is conscious before now, but now nearly all the commenters are saying that GLUT is conscious. What is the meaning of this?

Personally my response to the one would be similar to Caledonian's; perhaps more extreme. I think the linguistic analysis of philosophers is essentially worthless. Language is a means of communication and the referents a word has a matter of convention; meaning is a psychological property of no particular value. What concerns me is the person doing the communication. Where have they been and what have they done? You can, of course, follow the improbability on that. But my maxim is just,

Maxim: Language is a means of communication.

If somebody comes to you with just words; ignore them. Even if they're words about things. There isn't some metaphysical relation of reference reaching out from the noises leaving their mouth and connecting them to physical objects. There isn't some great Eternal Registry where "the problem of the chocolate cake in the asteroid belt" is suddenly registered as soon as somebody mentions the possibility of a chocolate cake in the asteroid belt. We do not, from that point on, have to solve the mystery of the chocolate cake or, and this is important, account for it in any way.

Philosophy and religion are very similar in their reverence for language. They both make the same essential mistake: they confer language with a power it does not have. It's the same view of language the shaman and the witchdoctor have. Language does something. It establishes something. The mere utterance of a word has some effect in this world or some other. For the shaman it's the spirit world; for the philosopher it's the non-actual possible world (or whatever happens to be in fashion this week). We know all this is not true as an empirical point of fact; language only effects the listener. This is why I reject philosophy outright.

Philosophy and religion are very similar in their reverence for language. They both make the same essential mistake: they confer language with a power it does not have.

I think you will find that philosophers are very well aware of the limitations of langage. More so than just about anyone else.

Language does something [..] for the philosopher it's the non-actual possible world.

Philosophers currently talk about PW's, but that does not mean they reify them. (What would non-actual mean)? And don't forget that some scientiss, and some less wrongers, and EY himself, do believe in alternate worlds. You need a better example. Or a different theory.

And while a Turing machine has a state register, this can be simulated by just using N lookup tables instead of one lookup table. It seems like we have to believe that 1), the mathematical structure of a UTM relative to a giant lookup table, which is very minimal indeed, is the key element required for consciousness, ...
TMs also have the notable ability to not halt for some inputs. And if you wanted to precompute those results, writing NULL values into your GLUT, I'd really like to know where the heck you got your Halting Oracle from. The mathematical structures are very different. For a UTM, the problem of whether it will halt for an arbitrary input is undecidable; in a GLUT with NULL values, you can just look up the input string and are done.

A GLUT AGI need not be a UTM, since most people have limited ability to execute programmes in their heads. You can write in "huh? I've lost track" for most answers to "what do you get if you execute the folowing programme steps".

Hi Caledonian. Hi Stephen. If I remember correctly, this is where the program that is the three of us having college bull sessions goes HALT and we never get any further, is it not? Once again, Eliezer says clearly what Caledonian was thinking and articulated through metaphor in one-on-one conversations (namely "Well, then it wouldn't be conscious. IMHO." ) but is predictably not understood by same, while I am far from sure. Eliezer: You don't know how much I wanted to see you type essentially the line "Ordinarily, when we're talking to a person, we tend to think that whatever is inside the skull, must be "where the consciousness is". It's only by playing Follow-The-Improbability that we can realize that the real source of the conversation we're having, is that-which-is-responsible-for the improbability of the conversation - however distant in time or space, as the Sun moves a wind-up toy.". Honestly, to me that summarizes the essence of not falling into the actual extremely common philosophical heresy of scientism, a heresy which I consider Chappell and Chalmers, for instance, to belong to (rather than positivism, which Chappell calls 'scientism' and which Caledonian doesn't actually believe in based on my personal communications with him despite his 'belief in belief' in it).

Not Conscious? I'd say the GLUT was not only conscious, it has god like powers. It can solve NP hard problems in one look up. It can prove anything in under a second.

It's easy for a human to confuse epsilon for zero. In most cases this would be a useful simplification, but a GLUT can take that simplification and use it against you. A look up table doesn't warp space and time? Well, actually it does, it's just that a normal one would warp it by an insignificant amount. We wouldn't normally think of a look up table as threatening a death star, but even a "small" GLUT of 10^500 entries has enough mass energy to destroy a death star from 10 billion light year away. Just by warping space and time!

Most arguments that involve a GLUT go something like this;
A GLUT is just a look up table and a look up table is obviously not ...
It's anything but obvious. A book is not conscious? How do you know? Maybe consciousness isn't a binary property, maybe we've just arbitrarily set a threshold, above that amount we call it conscious. A GLUT would have that in spades. Or maybe not. How can you be 100% confident that a look up table has zero consciousness when you don't even know for sure what consciousness is?

It can prove anything in under a second.

I may have missed the part where this is specified, but I imagine reading the GLUT would actually take longer than solving most problems, since it's unimaginably large.

Perhaps a GLUT cannot actually pass the Turing Test. Consider the following extension to the thought experiment.

I have a dilemma. I must conduct a Turing Test. I have two identical rooms. You will be in one room. A GLUT will be in the other. At the end of the experiment, I must destroy one of the two rooms. The Turing Test forbids me to peer inside the rooms, and I only communicate with simple textual question/responses.

What can I do to save your life? What I would want to do is create a window between the two rooms. It would allow all the information in each room to be visible to the other. I'm not sure if this illegitimately mutates the Turing Test or not, but it does seem to avoid violating the critical rule in the Turing Test that the experimenter must not peer into the room. I then ask one of the two rooms randomly: "Please give me a single question/response I should expect from the other room."

Assuming you are a rational person who actually wants to save your life, if I ask you this question, you will examine the GLUT, pick a single lookup, and give me the question/response. I will then ask the GLUT the question you gave me. The GLUT, being a helplessly deterministic lookup table, will have no option but to respond accordingly. I will then destroy the GLUT, and save your life. Conversely, if I ask the GLUT the question, I should expect that you - who wants to save your life, and who knows by looking through the window what the GLUT said you'd say, will answer anything other than what the GLUT said you would say. Either way, I can differentiate between you and the GLUT.

[Update: ciphergoth and FAW do a great job spotting the error in this intuition pump. To summarize, the GLUT, like you, can also include data from the window as input to its lookup table.]

I'm afraid this is just a misleading intuition pump. Eliezer has GLUT-reading powers, does he? Well, the GLUT has a body that it uses to type its responses in the Turing Test, and that body is capable of scanning the complete state of Eliezer's brain, from which the GLUT's enormous tables predict what he's going to say next.

When does the GLUT's scan occur? Before or after it has to start the Turing Test? If it does it beforehand, then it suffers predictability. But it can't do it afterwards, without ceasing to fit the definition of a lookup table.

The point I'm making is that the difference you're drawing between people and GLUTs isn't really to do with their essential nature: it's a more trivial asymmetry on things like how readable their state is and whether they have access to a private source of randomness. Fix these asymmetries and your problem case goes away.

Thanks ciphergoth; I updated the original comment to allude to the error you spotted.

A lookup table is stateless. The human is stateful. RAM beats ROM. This is not a trivial asymmetry but a fundamental asymmetry that enables the human to beat the GLUT. The algorithms:

Stateless GLUT:
Question 1 -> Answer 1
Question 2 -> Answer 2
Question 3 -> Answer 3
...

Stateful Human:
Any Question -> Any Answer other than what the GLUT said I'd say

The human's algorithm is bulletproof against answering predictably. The GLUT's algorithm can only answer predictably.

P.S. I wasn't entirely sure what you meant by "private source of randomness". I also apologize if I'm slow to grasp any of your points.

GLUT:

Task + Question + state of the human -> "Any Answer other than what the GLUT said I'd say"

If the human has looked up that particular output as well then that's another input for the GLUT, and since the table includes all possible inputs this possibility is included as well, to infinite recursion.

The problem for the GLUT is that the "state of the human" is a function of the GLUT itself (the window causes the recursion).

And the human has exactly the same problem.

You're right; got it. That's also what ciphergoth was trying to tell me when he said that the asymmetries could be melted away.

Thanks for update! By "private source of randomness" I mean one that's not available to the person on the other side of the window. Another way to look at it would be the sort of randomness you use to generate cryptographic keys - your adversary mustn't have access to the randomness you draw from it.

It depends on how you order it. With the natural numbers in ascending order, squares are less common. Interleaving them like {1, 2, 4, 3, 9, 5, 16, 6, 25, 7, ...}, they're equally common. With a different order type like {2, 3, 5, 6, 7, ..., 1, 4, 9, 16, 25, ...}, I have no idea. This is a problem.

See also Nick Bostrom's Infinite Ethics [PDF].

Psy-Kosh:

Does Tegmark provide any justification for the lower weight thing or is it a flat out "it could work if in some sense higher complexity realities have lower weight"?

It's the same justification as for the Kolmogorov prior: if you use a prefix-free code to generate random objects, less complex objects will come up more frequently. Descriptions of worlds with more tunable parameters must include those parameters, which adds complexity. (But, yes, if complexity/weight/frequency is ignored, there are infinitely more worlds above any complexity bound than below it.)

For that matter, what would it even mean for them to be lower weight?

Good question. With MWI, there's Robin's "mangled worlds" proposal (and maybe others) to generate objective frequencies; I don't know of any such suggestion for Tegmark's multiverse.

And any thoughts at all on why it seems like I'm not (at least, most of me seemingly isn't) a Boltzmann brain?

From Wikipedia: "Boltzmann proposed that we and our observed low-entropy world are a random fluctuation in a higher-entropy universe. Even in a near-equilibrium state, there will be stochastic fluctuations in the level of entropy. The most common fluctuations will be relatively small...." So we have strong evidence that this is false; there must be some reason to expect large, low-entropy universes to be more common than you would naively predict. Still, I would expect Boltzmann brains to outnumber 'normal' observers even within our universe, because there's only a narrow window of time for 'normal' observers to exist, but an infinity of heat death for Boltzmann brains to arise in, so I'm still confused.

Caledonian:

Well, the first point is to discard the idea that orderly perceptions are less probable than chaotic ones in the Dust.

Could be, but there doesn't seem to be any prior reason to suppose this. It seems that the dust should generate observer-moments with probability according to their algorithmic complexity, which would produce many more chaotic than normal ones. But it would solve the problem.

The second is to recognize that probability doesn't matter to the anthropic principle at all. You don't exist in the chaotic perspectives, so you never see them.

For every 'normal' possible world, there exist a huge number exactly like it but with small but glaring anomalies, like I have two sets of inconsistent memories or all coin flips come up heads or.... Observers could still exist in these partially-chaotic perspectives. There are also worlds that are almost entirely chaotic but with an island of order just big enough for one observer.

No one in particular: even if it's possible to account for why the dust wouldn't produce consciousness, the same arguments would still seem to apply to a non-conscious, purely computational Bayesian decision system (it would be surprised to observe order, etc.) I suspect this is actually a doubly wrong question, resulting from confusion about both consciousness and anthropic reasoning.

People who want to read more about this topic online may find that it is sometimes referred to as a "humongous" (slang for huge) lookup table or HLUT. Googling on that term will find some additional hits.

Psy-Kosh's point about implementations that use lookup tables internally of various sizes I think echos Moravec's point in Mind Children. The idea is that you could replace various sub-parts of your conscious AI with LUTs, ranging all the way from trivial substitutions up to a GLUT for the whole thing. Then as he says, when and where is the consciousness?

I would suggest that the answer is meaningless, that consciousness cannot necessarily be localized in the same way as some other properties. Where, after all, in our own brains, is the consciousness, if we zoom in and look at individual neurons? Is there a "consciousness scalar field" where we can indicate, at each point in the brain, how much consciousness is present there? I doubt it.

One other question this raises is the issue of implementation. There is an extensive philosophical debate (also involving Chalmers) on when a given system can be said to implement a given computation, in particular a conscious computation. I recall several years back Eliezer writing on these topics and at the time he saw this as a major stumbling block for functionalism. I would be interested in hearing how his thoughts have evolved, and I hope he can write about this soon.

Of course a GLUT can be conscious. A problem some may have with it would be that it is not self-modifying, for the table is set in stone, right? Well, consider it from this perspective:

First of all, I assume that all or some of the output is fed back into the input, directly or indirectly (or is that cheating? why?). Then, we can divide the GLUT in two parts, A and B, that differ only in one input: the fact that the "zombie" has previously heard a particular phrase, for example "You are not conscious, you ugly zombie!".

There is no need for the being to have any other kind of "memory" apart from the GLUT, because we can postulate that from the point that that phrase is heard, and produces an output in the "B" zone of the table, there is no possible combination of feedback plus external inputs that go out of the GLUT by tha "A" zone. With a truly G LUT, we can keep al the state we need.

Then we can easily say that the table has been "changed", for the outputs are coming from an entirely separated zone ("B") of the table, and it cannot go back to "A", so we might as well discard that part and say that the table has changed.

That misses my point. A process is needed to do the look-ups or the table just sits there.

Ah, I see you're not familiar with the works of Jorge Luis Borges. Permit me to hyperlink: The Library of Babel

I will step up and claim that GLUTs are conscious. Why wouldn't they be?

Because consciousness is precluded in the thought experiment. The whole idea is that the Zombie World is identical in every way - except it doesn't have this ephemeral consciousness thing.

Therefore the GLUT cannot be conscious, by the very design of the thought experiment it cannot be so. Yet there isn't any logical explanation for the behavior of the zombies without something, somewhere, that is conscious to drive them. That's why the GLUT came into the discussion in the first place - something has to tell the zombies what to do, and that something must be conscious (except it can't be, because the thought experiment precludes it).

Thus, an identical world without consciousness is inconceivable.

So does that mean a GLUT in the zombie world cannot be conscious, but a GLUT in our world (assuming infinite storage space, since apparently we were able to assume that for the zombie world) can be conscious?

Wow, a lot of things to say at this point.

Eliezer Yudkowsky: First, as I started reading, I was going to correct you and point out that Daniel Dennett thinks a GLUT can be conscious, as that is exactly his response to Searle's Chinese Room argument, thinking that I didn't need to read further. Fortunately, I did read the whole thing and find out, when I look at the substance of what the two of you believe, it's the same. While Dennett would say that the GLUT running in the Chinese Room is conscious, what you were really asking was, what is the source of the consciousness? Since that GLUT would have to be written by a consciousness, you two are in agreement.

Second, I don't think you have ruled out (shown to be low enough) the possibility of randomly picking out a GLUT that just happens to be conscious. While there is a low probability of picking just the right GLUT that happens to implement just the right lookup table, it's no different than any of the other unlikely things that had to happen for us to all be here. I mean, a certain group of people will point to the low probability of physical constants being just right/self-replicating molecules forming/single-celled organisms becoming multicellular/wing or flagellum or cell or blood clotting evolving, as evidence it couldn't have happened by chance (that there was a consciousness behind it). In response, one can just point to the anthropic principle -- why wouldn't that apply here? We could only be here to observe the universes where random processes grabbed that one GLUT that implemented something functionally similar to consciousness.

Finally, I had assumed through this series of posts that you were taking some position sharply divergent from Dennett. I mean, if the whole concept of qualia is incoherent, a universe lacking that incoherence isn't so impossible, right?

While there is a low probability of picking just the right GLUT that happens to implement just the right lookup table, it's no different than any of the other unlikely things that had to happen for us to all be here.

No. No. No. No. No.

The probability of picking the "just the right" GLUT is vastly smaller than any mere physical chain of events – there's no chance!

Hrm... as far as no one actually willing to jump in and say "a glut can be/is conscious"... What about Moravec and Egan? (Egan in Permutation City, Moravec in Simulation, Consciousness, Existance)... I don't recall them explicitly coming out and saying it, but it does seem to have been implied.

Anyways, I think I'm about to argue it... Or at least argue that there's something here that's seriously confusing me:

Okay, so you say that it's the generating process of the GLUT that has the associated consciousness, rather than the GLUT itself. Fine...

But exactly where is the breakdown between that and, say, the process that generates a human equivalent AI? Why not say that process is where the consciousness resides rather than the AI itself? if one takes at least some level of functionalism, allowing some optimizations and so on in the internal computations, then the internal "levers" can end up looking algorithmically very very different than the external, even if the behavior is identical.

In other words, as I start with the "correct" rods and levers to produce consciousness, then optimize various bits of it incramentally... when does the optimization process itself contain the majority of the consciousness?

More concretely, let's do something analogous to that hashlife program, creating a bunch of minigluts for clusters of neurons rather than a single superglut for the entire brain.

What's going on there? is the location of the consciousness now kinda spread out and scrambled in spacetime, a la Permutation City?

As we make the sizes of the clusers we're precomputing all possible states for larger, or basically grouping clusters and making megaclusters out of them... does the localization of the consciousness start to incrementally concentrate in spacetime toward the optimization process?

To perhaps make this really concrete.... implement turing machine in life universe, implement brain simulation on that, and then start with regular life simulation, then regular hashlife, and then incrementally "optimize" with larger and larger clusters of cells, so you end up with ever larger look up tables. ie, run the sim for a bit, then pause, do a step of optimization of the life CA algorithm (ie, life -> regular hashlife) run for a bit, pause, make a hashhashlife or make larger clusters, continue running, etc...)

This isn't so much an argument for a specific perspective so much as a thought experiment and question. I'm honestly not entirely sure how to view this. "Simplest" seems to be Permutation City style "scrambled in spacetime" consciousness.

How can you be 100% confident that a look up table has zero consciousness when you don't even know for sure what consciousness is?

Why not just define consciousness in a rational, unambiguous, non-contradictory way and then use it consistently throughout. If we are talking thought experiments here, it is up to us to make assumption(s) in our hypothesis. I don't recall EY giving HIS definition of consciousness for his thought experiment.

However, if the GLUT behaves exactly like a human, and humans are conscious, then by definition the GLUT is conscious, whatever that means.

Things that are true "by definition" are generally not very interesting.

If consciousness is defined by referring solely to behavior (which may well be reasonable, but is itself an assumption) then yes, it is true that something that behaves exactly like a human will be conscious IFF humans are conscious.

But what we are trying to ask, at the high level, is whether there is something coherent in conceptspace that partitions objects into "conscious" and "unconscious" in something that resembles what we understand when we talk about "consciousness," and then whether it applies to the GLUT. Demonstrating that it holds for a particular set of definitions only matters if we are convinced that one of the definitions in that set accurately captures what we are actually discussing.

If consciousness is defined by referring solely to behavior (which may well be reasonable, but is itself an assumption) then yes, it is true that something that behaves exactly like a human will be conscious IFF humans are conscious.

Thanx! This underscores the importance of our definitions. A hypothesis is an assumption or assumptions and the presenter needs to make their case, by defining the terms that are crucial to their hypothesis.

But what we are trying to ask, at the high level, is whether there is something coherent in conceptspace that partitions objects into "conscious" and "unconscious" in something that resembles what we understand when we talk about "consciousness," and then whether it applies to the GLUT.

A moving target is more difficult to hit than a stationary one! When we used human cells for conscious in referring to a GLUT, we instantly saw that we could throw that definition out. Now we can quickly move on to the next definition. To me conceptspace sounds like climbing the "what if" tree. I could spend all day going from branch to branch and never find the banana. I'll re-read those links you provided, in case I misunderstood what they are saying.

ADDED: I'm stuck in Thingspace when talking science and go to conceptspace when I am writing music or painting.

Why not just define consciousness in a rational, unambiguous, non-contradictory way and then use it consistently throughout.

If my goal is to talk about something with a particular definition, then I prefer not to use an existing word to refer to it when that word doesn't refer unambiguously to the definition I have in mind. That just leads to confusing conversations. I'd rather just make up a new term to go with my new made-up definition and talk about that.

Conversely, if my goal is to use the word "consciousness" in a way that respects the existing usage of the term, coming up with an arbitrary definition that is unambiguous and non-contradictory but doesn't respect that existing usage won't quite accomplish that. I mean, I could define "consciousness" as the ability to speak fluent English; that would be unambiguous and non-contradictory and there's even some historical precedent for it, but I consider it a poor choice of referent.

If my goal is to talk about something with a particular definition, then I prefer not to use an existing word to refer to it when that word doesn't refer unambiguously to the definition I have in mind. That just leads to confusing conversations. I'd rather just make up a new term to go with my new made-up definition and talk about that.

Well, casual conversation is not the same as using key terms (or words) in a scientific hypothesis, so that's a different subject, but new terms to define new ideas is fine if it's your hypothesis. In conversation, new definitions for old words would be confusing and defining old words in a new way could be confusing as well. That's not what I am saying.

Words can have multiple meanings and the dictionary gives the most popular usages. If we are appealing to the popular use then we still need to define the word. At any rate, whatever key terms that we use in our hypothesis must be precise, unambiguous, non circular, non-contradictory and used consistently throughout our presentation.

Conversely, if my goal is to use the word "consciousness" in a way that respects the existing usage of the term, coming up with an arbitrary definition that is unambiguous and non-contradictory but doesn't respect that existing usage won't quite accomplish that.

I'm saying it is important what EY meant by consciousness. If the person I quoted says we don't know what it is.....then that person doesn't know what the existing usage of the word is, or it is not well defined.

Anyways, why would you use a poor choice of a referent?

If my goal is to clarify some confusing aspects of what people think about when they use the word "consciousness", then if I end up talking about something other than what people think about when they use the word "consciousness" (for example, if I come up with some precise, unambiguous, non-circular, non-contradictory definition for the term) there's a good chance that I've lost sight of my goal.

Thanx! TheOtherDave:

The point of defining one's terms is to avoid confusion in the first place. It doesn't matter what anyone else thinks consciousness means. Only the meaning as defined in the theorist's hypothesis is important at this stage of the scientific method.

"there's a good chance that I've lost sight of my goal"

That's something I don't understand (with epistemic rationality- "The art of choosing actions that steer the future toward outcomes ranked higher in your preferences ").

This is fine when a person is making personal choices on how to act, but when it comes to knowledge (and especially the scientific method)....It seems like ultimately one would be interested in increasing one's understanding regardless of an individual's goals, preferences or values.

Oh well, at least we aren't using Weber's affectual rationality involving feelings here.

I would agree that if what I want to do is increase my understanding regardless of my ability to communicate effectively with other people (which isn't true of me, but might be true of others), and if communicating effectively with others doesn't itself contribute significantly to my understanding (which isn't true of me, but might be true of others), then choosing definitions for my words that maximize my internal clarity without reference to what those words mean to others is a pretty good strategy.

You started out by asking why EY doesn't do that, and I was suggesting that perhaps it's because his goals weren't the goals you're assuming here.

Reading between the lines a bit, I infer that the question was rhetorical in the first place, and your point is that maximizing individual understanding without reference to other goals, preferences, values, or communication with others should be what EY is doing... or perhaps that it is what he's doing, and he's doing a bad job of it.

If so, I apologize for misunderstanding.

@TheOtherDave:

Anotherblackhat said :

How can you be 100% confident that a look up table has zero consciousness when you don't even know for sure what consciousness is?

In response Monkeymind said :

Why not just define consciousness in a rational, unambiguous, non-contradictory way and then use it consistently throughout?

Not being100% confident what consciousness is, seemed to be a concern to anotherblackhat. Defining consciousness would have removed that concern.

No need to "read between the lines" as it was a straight forward question. I really didn't understand why the definition of consciousness wasn't laid out in advance of the thot experiment.

Defining terms allows one to communicate more effectively with others which is really important in any conversation but essential in presenting a hypothesis.

I was informed by Dlthomas that conceptspace is different than thingspace, so I think get the jest of it now.

However, my point was, and is, that the theorist's defs are crucial to the hypothesis and hypotheses don't care at all about goals, preferences, and values. Hypotheses simply illustrate the actors, define the terms in the script and set the stage for the first act. Now we can move on to the theory and hopefully form a conclusion.

No need to apologize, it is easy to misunderstand me, as I am not very articulate to begin with, and as usual, I don't understand what I know about it!

ADDED: And I still need to learn how to narrow the inferential gap!

Agreed that hypotheses don't care about goals, preferences, or values.
Agreed that for certain activities, well-defined terms are more important than anything else.

At any rate, whatever key terms that we use in our hypothesis must be precise, unambiguous, non circular, non-contradictory and used consistently throughout our presentation.

I'm personally okay with circular definition when used appropriately. For instance, there's the Haskell definition

naturalNumbers = 1 : (map (+ 1) naturalNumbers)

which tells you how to build the natural numbers in terms of the natural numbers.

Thanx! but, sorry, I don't know what that means. I do understand that numbers can 'do things' that physical objects can't. All words are concepts, but they resolve down to either concepts or objects. We have got to understand the difference between the two. Numbers are for conceptspace.

If I define "exist"- to be. It is circular. It tells us nothing. If a physicist says something exists in his hypothesis, he can not be circular in defining that term. He would have to define exist something like exist = matter + location.

If I choose to define consciousness as "self-aware" and everyone understands that to mean aware and aware of being aware, then I can proceed with my presentation. As long as I use the term consistently, everyone can see if my hypothesis is rational or not, then we can move on to the theory. If the theory explains the hypothesis then we can come to the conclusion that it is possible. We understand a better explanation may come along.

Anyways, I think probably the issue is that consciousness is what something does and not what something has.

However, if the GLUT behaves exactly like a human, and humans are conscious, then by definition the GLUT is conscious, whatever that means.

This seems to exactly contradict your first paragraph. What if I define "conscious" as "made of cells"?

If you don't know for sure what consciousness is, you define it as best you can, and proceed forward to see if your hypothesis is rational and that the theory is possible. If you define conscious as made of cells, then everyone knows right away a GLUT is not conscious (that is, if it is not made of cells) by YOUR def. and tells you, you are being irrational, please go back to the drawing board!

Part of the brain's function is to provide output to itself. Consequently, even though I would be quite happy saying C-3PO is conscious, I wouldn't be so quick to say that about a GLUT.

Still, it seems remarkable to me that everyone is treating consciousness as an either/or. Homo sapiens gradually became conscious after species that weren't. Infants gradually become conscious after a fertilized egg that was not. Let us put essentialism to rest.

And as an aside, I would state roughly that an organism is conscious iff it has theory of mind. That is, consciousness is ToM applied to oneself.

Surely the 'bottom line' is this:

Once you've described what a GLUT is and what it does, it's a mistake to think that there's anything more to be said about whether it's "really conscious". (Agreeing with Dennett against Chalmers:) consciousness isn't a fundamental property like electric charge but a 'woolly', 'high level' one like health or war. Clearly there's no reason to think that for every physical system, there is a well-defined answer to the question "is it healthy?" (or "is a war in progress?") You can