Eliezer, I suspect you are not being 100% honest here. I don't have any problems with a GLUT being conscious.

Hrm... as far as no one actually willing to jump in and say "a glut can be/is conscious"... What about Moravec and Egan? (Egan in Permutation City, Moravec in Simulation, Consciousness, Existance)... I don't recall them explicitly coming out and saying it, but it does seem to have been implied.

Anyways, I think I'm about to argue it... Or at least argue that there's something here that's seriously confusing me:

Okay, so you say that it's the generating process of the GLUT that has the associated consciousness, rather than the GLUT itself. Fine...

But exactly where is the breakdown between that and, say, the process that generates a human equivalent AI? Why not say that process is where the consciousness resides rather than the AI itself? if one takes at least some level of functionalism, allowing some optimizations and so on in the internal computations, then the internal "levers" can end up looking algorithmically very very different than the external, even if the behavior is identical.

In other words, as I start with the "correct" rods and levers to produce consciousness, then optimize various bits of it incramentally... when does the optimization proces... (read more)

Hi Caledonian. Hi Stephen. If I remember correctly, this is where the program that is the three of us having college bull sessions goes HALT and we never get any further, is it not? Once again, Eliezer says clearly what Caledonian was thinking and articulated through metaphor in one-on-one conversations (namely "Well, then it wouldn't be conscious. IMHO." ) but is predictably not understood by same, while I am far from sure. Eliezer: You don't know how much I wanted to see you type essentially the line "Ordinarily, when we're talking to... (read more)

"The GLUT is no more a zombie, than a cellphone is a zombie because it can talk about consciousness while being just a small consumer electronic device. The cellphone is just transmitting philosophy speeches from whoever happens to be on the other end of the line. A GLUT generated from an originally human brain-specification is doing the same thing."

You begin by saying that you are using "zombie" in a broader-than-usual sense, to denote something that "behave[s] exactly like a human without being conscious". The GLUT was con... (read more)

Isn't the state-space of similar such problems known to exceed the number of atoms in the Universe? There is a term for problems which are rendered unsolvable because there just isn't enough possible state-storing matter to represent them, but I can't think of it now.

Pardon me if this is a stupid question, my experience with AI is limited. Funny Eliezer should mention Haskell, I've got to get back to trying to wrap my brain around 'monads'.

The rule of the rationalist's game is that every improbable-seeming belief needs an equivalent amount of evidence to justify it.

Aren't you already breaking it allowing what you consider improbable GLUTs with no evidence?

Also how would you play this game with someone with a vastly different prior?

Any process can be replaced by a sufficiently-large lookup table with the right elements.

If you accept that a process can be conscious, you must acknowledge that lookup tables can be.

There is no alternative. Resistance is useless.

Let me be the first in this thread to suggest that, for the purposes of GLUTs, we should taboo the word "conscious." This post, in my opinion, is a shining example of Eliezer’s ability to verbally carve reality at its joints. After a remarkably clear discussion of the real problem, the question of “conscious” GLUTs seems like a silly near-boundary case.

Is there a technical reason I should think otherwise?

PK is right. I don't think a GLUT can be intelligent, since it can't remember what it's done. If you let it write notes in the sand and then use those notes as part of the future stimulus, then it's a Turing machine.

The notion that a GLUT could be intelligent is predicated on the good-old-fashioned AI idea that intelligence is a function that computes a response from a stimulus. This idea, most of us in this century now believe, is wrong.

"Any process can be replaced by a sufficiently-large lookup table with the right elements."

That misses my point. A process is needed to do the look-ups or the table just sits there.

If you abstract away the low-level details of how neurons work, couldn't the brain be considered a very large, multidimensional look-up table with a few rules regarding linkages and how to modify strengths of connections?

Phil: Gluts can certainly learn. A GLUT's program is this:

while (true) { x = sensory input y, z = GLUT(y, x) muscle control output = z }

Everything a GLUT has learned is encoded into y. Human GLUTS are so big that even their indices are huge.

Is the entity that results from gerrymandering together neural firings from different people's brains, so as to produce a pattern of neural firings similar to a brain but not corresponding to any "real person" in this Everett branch, conscious? How about gerrymandering together instructions occurring in different CPUs? Atomic motions in random rocks?

Consider a tiny look-up table mapping a few (input sentence, state) pairs to (output sentence, state) pairs - one small enough to practically be constructed, even. So long as you stick to the few sentences it accepts in the current state, it behaves exactly like a GLUT. If a GLUT is conscious, either this smaller table is conscious too, or it's the never activated entries that make the GLUT conscious.

"But suppose someone actually did reach into a GLUT-bin and by genuinely pure chance pulled out a GLUT that wrote philosophy papers?"

• this event has a probability that is best thought of as being smaller than 1 over the biggest number that most people can think of, so I don't particularly care.

That misses my point. A process is needed to do the look-ups or the table just sits there.

Ah, I see you're not familiar with the works of Jorge Luis Borges. Permit me to hyperlink: The Library of Babel

PK, Phil Goetz, and Larry D'Anna are making a crucial point here but I'm afraid it is somewhat getting lost in the noise. The point is (in my words) that lookup tables are a philosophical red herring. To emulate a human being they can't just map external inputs to external outputs. They also have to map a big internal state to the next version of that big external state. (That's what Larry's equations mean.)

If there was no internal state like this, a GLUT couldn't emulate a person with any memory at all. But by hypothesis, it does emulate a person (pe... (read more)

Internal state is not necessary. Consider a function f mapping strings to strings by means of a lookup table. Here are some examples of f evaluated with well-chosen inputs:

f("Hi, Dr. S here, how are you now that you're a lookup table?") = "Very well, thank you. I notice no difference."

f("Hi, Dr. S here, how are you now that you're a lookup table? Really, none at all?") = "Yes, really no differences at all."

f("Hi, Dr. S here, how are you now that you're a lookup table? You have insulted my entire family!") = "I know you well enough to know that my last reply could not possibly have insulted you; someone must be feeding me fake input histories again."

There should probably be timestamps in the input histories but that's an implementation detail. For what it's worth, I hold that f is conscious.

Of course a GLUT can be conscious. A problem some may have with it would be that it is not self-modifying, for the table is set in stone, right? Well, consider it from this perspective:

First of all, I assume that all or some of the output is fed back into the input, directly or indirectly (or is that cheating? why?). Then, we can divide the GLUT in two parts, A and B, that differ only in one input: the fact that the "zombie" has previously heard a particular phrase, for example "You are not conscious, you ugly zombie!".

There is no need ... (read more)

People who want to read more about this topic online may find that it is sometimes referred to as a "humongous" (slang for huge) lookup table or HLUT. Googling on that term will find some additional hits.

Psy-Kosh's point about implementations that use lookup tables internally of various sizes I think echos Moravec's point in Mind Children. The idea is that you could replace various sub-parts of your conscious AI with LUTs, ranging all the way from trivial substitutions up to a GLUT for the whole thing. Then as he says, when and where is the consc... (read more)

The more I think about it, the more I am convinced that if any GLUT could ever be made it would be an unspeakably horrible abomination. To explicitly represent the brain states of all the worst things that could happen to a person is a terrible thing. Weather the "internal state" variable is actually pointing at one doesn't seem to make a big moral difference. GLUTs are torture. They are the worst form of torture I've ever heard of. I'm glad they're almost certainly impossible.

I recall several years back Eliezer writing on these topics and at the time he saw this as a major stumbling block for functionalism. I would be interested in hearing how his thoughts have evolved, and I hope he can write about this soon.

Very, very strongly seconded.

Larry gives me another idea. Say the GLUT is implemented as a giant book with a person following instructions a la the Chinese Room. In the course of looking up the current (sentence, state) pair in the book, many other entries will inevitably impinge on the operator's retinas and enter their m... (read more)

Hal: Yeah, I actually am inclined toward thinking that something like Permutation City style cosmology/consciousness is actually valid... HOWEVER

If so, that seems to seperate consciousness and material reality to the point that one may as well say "what material reality?"

But then, one could say

"hrm, okay, so let's say that physics as we know it is the wrong reduction, and instead there's some other principle that ends up implying/producing consciousness, and something about that fundamental principle and so on causes statistical patterns/reg... (read more)

Greg Egan says, in the Permutation City FAQ:

I think the universe we live in provides strong empirical evidence against the “pure” Dust Theory, because it is far too orderly and obeys far simpler and more homogeneous physical laws than it would need to, merely in order to contain observers with an enduring sense of their own existence. If every arrangement of the dust that contained such observers was realised, then there would be billions of times more arrangements in which the observers were surrounded by chaotic events, than arrangements in which there were uniform physical laws.

Nick: oh, hey, cool, thanks. Didn't know about the existance of such a FAQ

Yeah, the uniformity thing (which I thought of in terms of existance of structure in experience) does seem to be a hit against it, and something I've spent time thinking about, still without conclusion though.

On the other hand, the chain of reasoning leading to it seems hard to argue against.

ie, what would have to be true for for something like the dust theory to be false? I have trouble thinking of any way of having the dust theory be false and yet also keeping anything like zombies... (read more)

Incidentally, I note that the uniformity/structure problem is also, near as I can tell, a hit against Tegmark style "all possible mathematical structures" multiverse

Not necessarily. Tegmark suggests that mathematical structures with higher algorithmic complexity [in what encoding?] have lower weight [is there a Mangled Worlds-like phenomenon that turns this weight into discrete objective frequencies?], and that laws producing an orderly universe have lower complexity than chaotic universes or especially encodings of specific chaotic experiences.

Does Tegmark provide any justification for the lower weight thing or is it a flat out "it could work if in some sense higher complexity realities have lower weight"?

For that matter, what would it even mean for them to be lower weight?

I'd, frankly, expect the reverse. The more "tunable parameters", the more patterns of values they could take on, so...

For that matter, if some means of different weights/measures/whatever could be applied to the different algorithm's, why disallow that sort of thing being applied to different "dust interpretations"?

And any thoughts at all on why it seems like I'm not (at least, most of me seemingly isn't) a Boltzmann brain?

Well, the first point is to discard the idea that orderly perceptions are less probable than chaotic ones in the Dust.

The second is to recognize that probability doesn't matter to the anthropic principle at all. You don't exist in the chaotic perspectives, so you never see them.

Psy-Kosh:

Does Tegmark provide any justification for the lower weight thing or is it a flat out "it could work if in some sense higher complexity realities have lower weight"?

It's the same justification as for the Kolmogorov prior: if you use a prefix-free code to generate random objects, less complex objects will come up more frequently. Descriptions of worlds with more tunable parameters must include those parameters, which adds complexity. (But, yes, if complexity/weight/frequency is ignored, there are infinitely more worlds above any complexit... (read more)

Psy-Kosh : "Yeah, the uniformity thing (which I thought of in terms of existance of structure in experience) does seem to be a hit against it, and something I've spent time thinking about, still without conclusion though.

On the other hand, the chain of reasoning leading to it seems hard to argue against.

ie, what would have to be true for for something like the dust theory to be false? I have trouble thinking of any way of having the dust theory be false and yet also keeping anything like zombies disallowed."

Psy-Kosh, that isn't a chain of reasoni... (read more)

It seems that the dust should generate observer-moments with probability according to their algorithmic complexity, which would produce many more chaotic than normal ones.

The full version of the Library of Babel can be generated by "walking" through the versions with a limited number of texts, each of finite length. It contains every possible string that can be composed of a given set of symbols - infinitely many strings, each infinitely long. Any finite string that can appear in the Library, does appear - infinitely many times.

In the Englis... (read more)

It's interesting that Eliezer never heard anyone say that a GLUT is conscious before now, but now nearly all the commenters are saying that GLUT is conscious. What is the meaning of this?

Unknown: I was unclear. I meant "rejecting the assumptions involved in the chain of reasoning that leads to the dust hypothesis would seem to require accepting things very much like zombies, and in ways that seem rather preposterous, at least to me"

Yes, obviously if ~zombie -> dust, then ~dust->zombie. Either way, I know I'm very confused about this whole matter.

Caledonian: Yes, AB will be more common than CDEFG as a substring. but ABABABABABAB will be less common than AB(insert-random-sequence-here)

In other words, the number of "me&qu... (read more)

In the FULL version, "AB" and "CDEFG" are equally probable. Each appears infinitely often, but the order of the category of infinities that they belong to is the same.

Would you argue that odd numbers are as probable as even numbers in the set of natural numbers, because the order of the category of infinities that they belong to is the same?

How about squares (1, 2, 4, 9, 16, ...) versus non-square numbers? Prime numbers versus composite numbers?

It depends on how you order it. With the natural numbers in ascending order, squares are less common. Interleaving them like {1, 2, 4, 3, 9, 5, 16, 6, 25, 7, ...}, they're equally common. With a different order type like {2, 3, 5, 6, 7, ..., 1, 4, 9, 16, 25, ...}, I have no idea. This is a problem.

See also Nick Bostrom's Infinite Ethics [PDF].

Aw, Nick, you spoiled the punchline! ;-)

Would you argue that odd numbers are as probable as even numbers in the set of natural numbers, because the order of the category of infinities that they belong to is the same? How about squares (1, 2, 4, 9, 16, ...) versus non-square numbers? Prime numbers versus composite numbers?

As far as I understand, the sets of odd numbers, squares, and primes are all countable.

As such, a one-to-one correspondence can be established between them and the counting numbers. Therefore, considered across infinity, there are just as many primes as there are odd numbers... (read more)

Caledonian,

The part I have a problem with is where you go from the cardinality of the sets to a judgment of "equally probable".

Let me put it this way: you wrote,

In the English version, in any of the truncated (and sufficiently long) versions of the Library, the sequence "AB" is much more common than "CDEFG". It doesn't matter whether the texts are ten thousand letters long, or ten billion - the first is less complex and thus more probable than the second.

The "any" is the problem. I can construct a truncated versio... (read more)

My statement doesn't hold in ANY truncated version of the Library - it's not difficult to construct an example, because any finite version automatically serves.

But we're not DEALING with a finite version of the Library. We are dealing with the infinite version. And infinity wreaks some pretty serious havoc on conventional concepts of probability.

So why do you say that all sentences have equal probability, rather than that the probability is undefined, which would seem to be the default option?

Some of those instructions won't halt, so eventually you'll get hung up in an infinite loop without outputting anything. And the Halting Problem has no general solution...

a "logically possible" but fantastic being — a descendent of Ned Block's Giant Lookup Table fantasy...

First, I haven't seen how this figures into an argument, and I see that Eliezer has already taken this in another direction, but...

What immediately occurs to me is that there's a big risk of a faulty intuition pump here. He's describing, I assume, a lookup table large enough to describe your response to every distinguishable sensory input you could conceivably experience during your life. The number of entries is unimaginable. But I sus... (read more)

Cyan: not true. As you can see, the non-halting processes don't prevent the others from running; they slow them down, but who cares when you have an infinite computer?

Tom: what do you think of my previous comment about a tiny look-up table?

As far as infinities, well, I think I'll for now stick with the advice of only bringing in infinities via well defined limits unless absolutely needed otherwise.

That's a good strategy and I recommend you stick to it.

The infinities are absolutely needed, here.

Caledonian: But do we here need to go beyond "well behaved limit defined infinities"?

Nick, you're right. I just misread/misinterpreted the pseudo-code "emulate_one_instruction".

Caledonian: But do we here need to go beyond "well behaved limit defined infinities"?

You do if you want to talk about certain sets. Some of those sets are relevant to the Dust hypothesis. Therefore, if you want to talk about the Dust hypothesis, you have to be willing to discuss infinities in a more complex way.

Short answer: yes.

Paul and Patricia Churchland, and Jerry Fodor, and others, have argued that GLUTs would be conscious.

They would be conscious. But they need memory, because the past provides context that changes proper responses to future questions / dialogue.

Amendment: I said GLUTs need memory based on the idea of perfectly duplicating the behavior of some other conscious being, like Eliezer, who does have memory. But there are brain-damaged people with various deficiencies in long- and/or short-term memory who still have conscious experience, so a GLUT without the ability to store new memories could be conscious like those people. Anyhoo.

A person's thoughts are underdetermined by their actions - there's no way, probably even in principle, to know nearly as much about my current thoughts as I do by observing my macro-level behavior (as opposed to micro-scale heat/EM wave output), and definitely no way to do so by observing what I type, even over a long period of interaction. So, since a GLUT is purely behavioral, which of the many possible experiences corresponding to my behavior would arise from a GLUT simulating me?

Nick: a GLUT wouldn't just be a list of actions though, it'd be a list, basically, of all possible outputs for all possible inputs.

In other words, if I simply knew your actions, that may underdetermine you, but if I knew all the ways you would have acted for all possible circumstances, well, it's not obvious to be that that would underdetermine you.

It seems likely to me that even that, for reasonable definitions of "action", couldn't distinguish between e.g. me and a very good improviser with a rich model of my mind (and running at a high subjective speedup) but completely different private thoughts, or a group of such people, or between me and me plus some secret thought I would never tell anyone or act on but regularly think about.

Nick: Are you even reasonably confident that such an impostor wouldn't, effectively, have instanstiated a version of you in their head?

Even if they did (and I doubt they would have to, but am less confident), they would also have thoughts that weren't mine.

I'm sure this will come across as naïve or loony, but is anyone else here occasionally terrified by the idea that they might 'wake up' as a Boltzmann brain at some point, with a brain arranged in such a way as to subject them to terrible agony?

Part of the brain's function is to provide output to itself. Consequently, even though I would be quite happy saying C-3PO is conscious, I wouldn't be so quick to say that about a GLUT.

Still, it seems remarkable to me that everyone is treating consciousness as an either/or. Homo sapiens gradually became conscious after species that weren't. Infants gradually become conscious after a fertilized egg that was not. Let us put essentialism to rest.

And as an aside, I would state roughly that an organism is conscious iff it has theory of mind. That is, consciousness is ToM applied to oneself.

A GLUT consciousness would need to store an internal state for the consciousness it is modeling. This could be as detailed as the region of configuration space describing an equivalent brain. You have a mapping from (sensation, state) to (external output, state). Since this is essentially a precomputed physical simulation, it's trivially capable of consciousness.

Eliminating the state parameter would lead to non-consciousness.

"Follow The Improbability" is a wonderful thing. Thank you.

As far as I can tell, GLUTs have to fail Turing tests for relativistic reasons.

Presumably lookup tables need to be stored somewhere in the universe. The number of possible lookups a GLUT might have to do to respond to whatever's happened in a Turing test so far grows exponentially with time, so the distance information has to travel from some part of the lookup table to an output device also grows exponentially with time (and Grover's algorithm doesn't change this). Since the information can't travel faster than the speed of light, before long a tester wo... (read more)

'a "logically possible" but fantastic being' [Dennett]

I don't see where the top posting is going on the whole. P-Zombies are always supposed to logically possible, as Dennet says. There may be a lot of things wrong with logical possibility: it may be imposssible to derive real-world consequences from it, it may not exist..but whatever it is, it is not a level of probablity, even a small one. Tell a zombiephile that p-zombies are highly unlikey, and she'll reply "sure, but they're still logically possible".

GLUTs pose a challenge to t... (read more)

In this line of business you meet an awful lot of people who think that an arbitrarily generated powerful AI will be "moral".

A good counter to this argument would be to find a culture with morals strongly opposed to our own, and demonstrate that it is logical and internally consistent. My inability to think of such a culture could be interpreted as evidence that a sufficiently-powerful AI would be moral. But I think it's more likely that the morals we agree on are properties common to most moral frameworks that are workable in our particular b... (read more)

Probably I am not right, but it looks to me that consciousness can go on without any "inputs" and "outputs". If I sit in the dark room alone and think about some sort of problem then I neither taking any inputs at that moment and nor generating any output uless I decide to think aloud :) So if you believe I am not a zombie then I am consciousness regardless if there are any inputs/outputs.

One of the best examples of the GLUT which I used to find very convincing, is by Jaron Lanier. (https://youtu.be/RgfFFRFPvyw) Instead of randomly pulling a computer out of nowhere, it's just the finite set of all possible computers. He uses this not to argue for zombies, but to introduce confusion and show how since hailstorms and asteroids can't be conscious, nobody really knows what they're talking about, therefore dualism is just as valid as reductionism. I now see where the error in reasoning is, thanks.