New Comment
I quite liked the way that this post presented your intellectual history on the topic, it was interesting to read to see where you're coming from.
That said, I didn't quite understand your conclusion. Starting from Chap. 7, you seem to be saying something like, "everyone has a different definition for what consciousness is; if we stop treating consciousness as being a single thing and look at each individual definition that people have, then we can look at different systems and figure out whether those systems have those properties or not".
This makes sense, but - as I think you yourself said earlier in the post - the hard problem isn't about explaining every single definition of consciousness that people might have? Rather it's about explaining one specific question, namely:
The explanatory gap in the philosophy of mind, represented by the cross above, is the difficulty that physicalist theories seem to have in explaining how physical properties can give rise to a feeling, such as the perception of color or pain.
You cite Critch's list of definitions people have for consciousness, but none of the three examples that you quoted seem to be talking about this property, so I don't see how they're related or why you're bringing them up.
With regard to this part:
If they do reinvent the hard problem, it would be a big sign that the AIs in the simulation are “conscious” (in the reconstructed sense).
I assert that this experiment would solve the hard problem, because we could look at the logs,[4] and the entire causal history of the AI that utters the words "Hard pro-ble-m of Con-scious-ness" would be understandable. Everything would just be plainly understandable mechanistically, and David Chalmer would need to surrender.
This part seems to be quite a bit weaker than what I read you to be saying earlier. I interpreted most of the post to be saying "I have figured out the solution to the problem and will explain it to you". But this bit seems to be weakening it to "in the future, we will be able to create AIs that seem phenomenally conscious and solve the hard problem by looking at how they became that". Saying that we'll figure out an answer in the future when we have better data isn't actually giving an answer now.
Thank you for the kind words!
Saying that we'll figure out an answer in the future when we have better data isn't actually giving an answer now.
Okay, fair enough, but I predict this would happen: in the same way that AlphaGo rediscovered all of chess theory, it seems to me that if you just let the AIs grow, you can create a civilization of AIs. Those AIs would have to create some form of language or communication, and some AI philosopher would get involved and then talk about the hard problem.
I'm curious how you answer those two questions:
- Let's say we implement this simulation in 10 years and everything works the way I'm telling you now. Would you update?
- What is the probability that this simulation is possible at all?
If you expect to update in the future, just update now.
To me, this thought experiment solves the meta-problem and so dissolves the hard problem.
- Let's say we implement this simulation in 10 years and everything works the way I'm telling you now. Would you update?
Well, it's already my default assumption that something like this would happen, so the update would mostly just be something like "looks like I was right".
2. What is the probability that this simulation is possible at all?
You mean one where AIs that were trained with no previous discussion of the concept of consciousness end up reinventing the hard problem on their own? 70% maybe.
If you expect to update in the future, just update now.
That sounds like it would violate conservation of expected evidence:
... for every expectation of evidence, there is an equal and opposite expectation of counterevidence.
If you expect a strong probability of seeing weak evidence in one direction, it must be balanced by a weak expectation of seeing strong evidence in the other direction. If you’re very confident in your theory, and therefore anticipate seeing an outcome that matches your hypothesis, this can only provide a very small increment to your belief (it is already close to 1); but the unexpected failure of your prediction would (and must) deal your confidence a huge blow. On average, you must expect to be exactly as confident as when you started out. Equivalently, the mere expectation of encountering evidence—before you’ve actually seen it—should not shift your prior beliefs.
To me, this thought experiment solves the meta-problem and so dissolves the hard problem.
I don't see how it does? It just suggests that a possible approach by which the meta-problem could be solved in the future.
Suppose you told me that you had figured out how to create cheap and scalable source of fusion power. I'd say oh wow great! What's your answer? And you said that, well, you have this idea for a research program that might, in ten years, produce an explanation of how to create cheap and scalable fusion power.
I would then be disappointed because I thought you had an explanation that would let me build fusion power right now. Instead, you're just proposing another research program that hopes to one day achieve fusion power. I would say that you don't actually have it figured it out yet, you just think you have a promising lead.
Likewise, if you tell me that you have a solution to the meta-problem, then I would expect an explanation that lets me understand the solution to the meta-problem today. Not one that lets me do it ten years in the future, when we investigate the logs of the AIs to see what exactly it was that made them think the hard problem was a thing.
I also feel like this scenario is presupposing the conclusion - you feel that the right solution is an eliminativist one, so you say that once we examine the logs of the AIs, we will find out what exactly made them believe in the hard problem in a way that solves the problem. But a non-eliminativist might just as well claim that once we examine the logs of the AIs, we will eventually be forced to conclude that we can't find an answer there, and that the hard problem still remains mysterious.
Now personally I do lean toward thinking that examining the logs will probably give us an answer, but that's just my/your intuition against the non-eliminativist's intuition. Just having a strong intuition that a particular experiment will prove us right isn't the same as actually having the solution.
hmm, I don't understand something, but we are closer to the crux :)
You say:
- To the question, "Would you update if this experiment is conducted and is successful?" you answer, "Well, it's already my default assumption that something like this would happen".
- To the question, "Is it possible at all?" You answer 70%.
So, you answer 99-ish% to the first question and 70% to the second question, this seems incoherent.
It seems to me that you don't bite the bullet for the first question if you expect this to happen. Saying, "Looks like I was right," seems to me like you are dodging the question.
That sounds like it would violate conservation of expected evidence:
Hum, it seems there is something I don't understand; I don't think this violates the law.
I don't see how it does? It just suggests that a possible approach by which the meta-problem could be solved in the future.
I agree I only gave the skim of the proof, it seems to me that if you can build the pyramid, brick by brick, then this solved the meta-problem.
for example, when I give the example of meta-cognition-brick, I say that there is a paper that already implements this in an LLM (and I don't find this mysterious because I know how I would approximately implement a database that would behave like this).
And it seems all the other bricks are "easily" implementable.
hmm, I don't understand something, but we are closer to the crux :)
Yeah I think there's some mutual incomprehension going on :)
- To the question, "Would you update if this experiment is conducted and is successful?" you answer, "Well, it's already my default assumption that something like this would happen".
- To the question, "Is it possible at all?" You answer 70%.
So, you answer 99-ish% to the first question and 70% to the second question, this seems incoherent.
For me "the default assumption" is anything with more than 50% probability. In this case, my default assumption has around 70% probability.
It seems to me that you don't bite the bullet for the first question if you expect this to happen. Saying, "Looks like I was right," seems to me like you are dodging the question.
Sorry, I don't understand this. What question am I dodging? If you mean the question of "would I update", what update do you have in mind? (Of course, if I previously gave an event 70% probability and then it comes true, I'll update from 70% to ~100% probability of that event happening. But it seems pretty trivial to say that if an event happens then I will update to believing that the event has happened, so I assume you mean some more interesting update.)
Hum, it seems there is something I don't understand; I don't think this violates the law.
I may have misinterpreted you; I took you to be saying "if you expect to see this happening, then you might as well immediately update to what you'd believe after you saw it happen". Which would have directly contradicted "Equivalently, the mere expectation of encountering evidence—before you’ve actually seen it—should not shift your prior beliefs".
I agree I only gave the skim of the proof, it seems to me that if you can build the pyramid, brick by brick, then this solved the meta-problem.
for example, when I give the example of meta-cognition-brick, I say that there is a paper that already implements this in an LLM (and I don't find this mysterious because I know how I would approximately implement a database that would behave like this).
Okay. But that seems more like an intuition than even a sketch of a proof to me. After all, part of the standard argument for the hard problem is that even if you explained all of the observable functions of consciousness, the hard problem would remain. So just the fact that we can build individual bricks of the pyramid isn't significant by itself - a non-eliminativist might be perfectly willing to grant that yes, we can build the entire pyramid, while also holding that merely building the pyramid won't tell us anything about the hard problem nor the meta-problem. What would you say to them to convince them otherwise?
Thank you for clarifying your perspective. I understand you're saying that you expect the experiment to resolve to "yes" 70% of the time, making you 70% eliminativist and 30% uncertain. You can't fully update your beliefs based on the hypothetical outcome of the experiment because there are still unknowns.
For myself, I'm quite confident that the meta-problem and the easy problems of consciousness will eventually be fully solved through advancements in AI and neuroscience. I've written extensively about AI and path to autonomous AGI here, and I would ask people: "Yo, what do you think AI is not able to do? Creativity? Ok do you know....". At the end of the day, I would aim to convince them that anything humans are able to do, we can reconstruct everything with AIs. I'd put my confidence level for this at around 95%. Once we reach that point, I agree I think it will become increasingly difficult to argue that the hard problem of consciousness is still unresolved, even if part of my intuition remains somewhat perplexed. Maintaining a belief in epiphenomenalism while all the "easy" problems have been solved is a tough position to defend - I'm about 90% confident of this.
So while I'm not a 100% committed eliminativist, I'm at around 90% (when I was at 40% in chapter 6 in the story). Yes, even after considering the ghost argument, there's still a small part of my thinking that leans towards Chalmers' view. However, the more progress we make in solving the easy and meta-problems through AI and neuroscience, the more untenable it seems to insist that the hard problem remains unaddressed.
a non-eliminativist might be perfectly willing to grant that yes, we can build the entire pyramid, while also holding that merely building the pyramid won't tell us anything about the hard problem nor the meta-problem.
I actually think a non-eliminativist would concede that building the whole pyramid does solve the meta-problem. That's the crucial aspect. If we can construct the entire pyramid, with the final piece being the ability to independently rediscover the hard problem in an experimental setup like the one I described in the post, then I believe even committed non-materialists would be at a loss and would need to substantially update their views.
The meta problem of consciousness is about explaining why people think they are conscious.
Even if we get such a result with AIs where AIs invent a concept like consciousness from scratch, that would only tell us that they also think they have sth that we call consciousness, but not yet why they think this.
That is, unless we can somehow precisely inspect the cognitive thought processes that generated the consciousness concept in AIs, which on anything like the current paradigm we won't be.
Another way to frame it: Why would it matter that an AI invents the concept of consciousness, rather than another human? Where is the difference that lets us learn more about the hard/meta problem of consciousness in the first place?
Separately, even if we could analyze the thought processes of AIs in such a case so we would solve the meta problem of consciousness by seeing explanations of why AIs/people talk about consciousness the way they do, that doesn't mean you already have solved the meta-problem of consciousness now.
Aka just because you know it's solvable doesn't mean you're done. You haven't solved it yet. Just like the difference between knowing that general relativity exists and understanding the theory and math.
So I've been trying to figure out whether or not to chime in here, and if so, how to write this in a way that doesn't come across as combative. I guess let me start by saying that I 100% believe your emotional struggle with the topic and that every part of the history you sketch out is genuine. I'm just very frustrated with the post, and I'll try to explain why.
It seems like you had a camp #2 style intuition on Consciousness (apologies for linking my own post but it's so integral to how I think about the topic that I can't write the comment otherwise), felt pressure to deal with the arguments against the position, found the arguments against the position unconvincing, and eventually decided they are convincing after all because... what? That's the main thing that perplexes me; I don't understand what changed. The case you lay out at the end just seems to be the basic argument for illusionism that Dennett et al have made over 20 years ago.
This also ties in with a general frustration that's not specific to your post; the fact that we can't seem to get beyond the standard arguments for both sides is just depressing to me. There's no semblance of progress on this topic on LW in the last decade.
You mentioned some theories of consciousness, but I don't really get how they impacted your conclusion. GWT isn't a camp #2 proposal at all as you point out. IIT is one but I don't understand your reasons for rejection -- you mentioned that it implies a degree of panpsychism, which is true, but I believe that shouldn't affect its probability one way or another?[1] (I don't get the part where you said that we need a threshold; there is no threshold for minimal consciousness in IIT.) You also mention QRI but don't explain why you reject their approach. And what about all the other theories? Like do we have any reason to believe that the hypothesis space is so small that looking at IIT, even if you find legit reasons to reject it, is meaningful evidence about the validity of other ideas?
If the situation is that you have an intuition for camp #2 style consciousness but find it physically implausible, then there's be so many relevant arguments you could explore, and I just don't see any of them in the post. E.g., one thing you could do is start from the assumption that camp #2 style consciousness does exist and then try to figure out how big of a bullet you have to bite. Like, what are the different proposals for how it works, and what are the implications that follow? Which option leads to the smallest bullet, and is that bullet still large enough to reject it? (I guess the USA being conscious is a large bullet, but why is that so bad, and what the approaches that avoid the conclusion, and how bad are they? Btw IIT predicts that the USA is not conscious.) How does consciousness/physics even work on a metaphysical level; I mean you pointed out one way it doesn't work, which is epiphenomenalism, but how could it work?
Or alternatively, what are the different predictions of camp #2 style consciousness vs. inherently fuzzy, non-fundamental, arbitrary-cluster-of-things-camp-#1 consciousness? What do they predict about phenomenology or neuroscience? Which model gets more Bayes points here? They absolutely don't make identical predictions!
Wouldn't like all of this stuff be super relevant and under-explored? I mean granted, I probably shouldn't expect to read something new after having thought about this problem for four years, but even if I only knew the standard arguments on both sides, I don't really get the insight communicated in this post that moved you from undecided or leaning camp #2 to accepting the illusionist route.
The one thing that seems pretty new is the idea that camp #2 style consciousness is just a meme. Unfortunately, I'm also pretty sure it's not true. Around half of all people (I think slightly more outside of LW) have camp #2 style intuitions on consciousness, and they all seem to mean the same thing with the concept. I mean they all disagree about how it works, but as far as what it is, there's almost no misunderstanding. The talking past each other only happens when camp #1 and camp #2 interact.
Like, the meme hypothesis predicts that the "understanding of the concept" spread looks like this:
but if you read a lot of discussions, LessWrong or SSC or reddit or IRL or anywhere, you'll quickly find that it looks like this:
Another piece of the puzzle is the blog post by Andrew Critch: Consciousness as a conflationnary alliance term. In summary, consciousness is a very loaded/bloated/fuzzy word, people don't mean the same thing when talking about it.
This shows that if you ask camp #1 people -- who don't think there is a crisp phenomenon in the territory for the concept -- you will get many different definitions. Which is true but doesn't back up the meme hypothesis. (And if you insist in a definition, you can probably get camp #2 people to write weird stuff, too. Especially if you phrase it in such a way that they think they have to point to the nearest articulate-able thing rather than gesture at the real thing. You can't just take the first thing people about this topic say without any theory of mind and take it at face value; most people haven't thought much about the topic and won't give you a perfect articulation of their belief.)
So yeah idk, I'm just frustrated that we don't seem to be getting anywhere new with this stuff. Like I said, none of this undermines your emotional struggle with the topic.
We know probability consists of Bayesian Evidence and prior plausibility (which itself is based on complexity). The implication that IIT implies panpsychism doesn't seem to affect either of those -- it doesn't change the prior of IIT since IIT is formalized so we already know its complexity, and it can't provide evidence one way or another since it has no physical effect. (Fwiw I'm certain that IIT is wrong, I just don't think the panpsychism part has anything to do with why.) ↩︎
1Thanks for jumping in! And I'm not that emotionally struggling with this, this was more of a nice puzzle, so don't worry about it :)
I agree my reasoning is not clean in the last chapter.
To me, the epiphany was that AI would rediscover everything like it rediscovered chess alone. As I've said in the box, this is a strong blow to non-materialistic positions, and I've not emphasized this enough in the post.
I expect AI to be able to create "civilizations" (sort of) of its own in the future, with AI philosophers, etc.
Here is a snippet of my answer to Kaj, let me know what you think about it:
I'm quite confident that the meta-problem and the easy problems of consciousness will eventually be fully solved through advancements in AI and neuroscience. I've written extensively about AI and path to autonomous AGI here, and I would ask people: "Yo, what do you think AI is not able to do? Creativity? Ok do you know....". At the end of the day, I would aim to convince them that anything humans are able to do, we can reconstruct everything with AIs. I'd put my confidence level for this at around 95%. Once we reach that point, I agree I think it will become increasingly difficult to argue that the hard problem of consciousness is still unresolved, even if part of my intuition remains somewhat perplexed. Maintaining a belief in epiphenomenalism while all the "easy" problems have been solved is a tough position to defend - I'm about 90% confident of this.
If the Turing thesis is correct, AI can, in principle, solve every problem a human can solve. I don't doubt the Turing thesis and hence would assign over 99% probability to this claim:
At the end of the day, I would aim to convince them that anything humans are able to do, we can reconstruct everything with AIs.
(I'm actually not sure where your 5% doubt comes from -- do you assign 5% on the Turing thesis being false, or are you drawing a distinction between practically possible and theoretically possible? But even then, how could anything humans do be practically impossible for AIs?)
But does this prove eliminativism? I don't think so. A camp #2 person could simply reply something like "once we get a conscious AI, if we look at the precise causal chain that leads it to claim that it is conscious, we would understand why that causal chain also exhibits phenomenal consciousness".
Also, note that among people who believe in camp #2 style consciousness, almost all of them (I've only ever encountered one person who disagreed) agree that a pure lookup table would not be conscious. (Eliezer agrees as well.) This logically implies that camp #2 style consciousness is not about ability to do a thing, but rather about how that thing is done (or more technically put, it's not about the input/output behavior of a system but an algorithmic or implementation-level description). Equivalently, it implies that for any conscious algorithm , there exists a non-conscious algorithm with identical input/output behavior (this is also implied by IIT). Therefore, if you had an AI with a certain capability, another way that a camp #2 person could respond is by arguing that you chose the wrong algorithm and hence the AI is not conscious despite having this capability. (It could be the case that all unconscious implementations of the capability are computationally wasteful like the lookup table and hence all practically feasible implementations are conscious, but this is not trivially true, so you would need to separately argue for why you think this.)
Maintaining a belief in epiphenomenalism while all the "easy" problems have been solved is a tough position to defend - I'm about 90% confident of this.
Epiphenomenalism is a strictly more complex theory than Eliminativism, so I'm already on board with assigning it <1%. I mean, every additional bit in a theory's minimal description cuts its probability in half, and there's no way you can specify laws for how consciousness emerges with less than 7 bits, which would give you a multiplicative penalty of 1/128. (I would argue that because Epiphenomenalism says that consciousness has no effect on physics and hence no effect on what empirical data you receive, it is not possible to update away from whatever prior probability you assign to it and hence it doesn't matter what AI does, but that seems beside the point.) But that's only about Epiphenomenalism, not camp #2 style consciousness in general.
Maybe I am hopelessly ill, but I think there is a problem in chapter 7:
"The quest to unravel the mystery of consciousness involves not just defining the term but reconstructing the entire causal chain that leads some people to speak about it, why we utter the word 'con-scious-ness'—to understand why, for example, my lips articulate this word."
Yes, it is possible to explain all the processes in my brain that cause my body to produce the sounds "why I have consciousness," but it will not answer the main question: Why are these processes in the brain happening from the first-person perspective? Why is anything in this Universe happening from the first-person perspective?
From my point of view, the simplest answer to this question is that all the processes in this world are in some way happening from the first-person perspective; that feelings are not produced by matter, but instead what we call "matter" and "feelings" are just two different aspects of the same substance. Yes, it may sound weird and does not fully solve the problem, but at least we no longer have to wonder what mysterious magic is happening inside our brains that produces the subject.
I've traveled these roads too. At some point I thought that the hard problem reduced to the problem of deriving an indexical prior, a prior on having a particular position in the universe, which we should expect to derive from specifics of its physical substrate, and it's apparent that whatever the true indexical prior is, it can't be studied empirically, it is inherently mysterious. A firmer articulation of "why does this matter experience being". Today, apparently, I think of that less as a deeply important metaphysical mystery and more just as another imperfect logical machine that we have to patch together just well enough to keep our decision theory working. Last time I scratched at this I got the sense that there's really no truth to be found beyond that. IIRC Wei Dai's UDASSA answers this with the inverse kolmogorov complexity of the address of the observer within the universe, or something. It doesn't matter. It seems to work.
But after looking over this, reexamining, yeah, what causes people to talk about consciousness in these ways? And I get the sense that almost all of the confusion comes from the perception of a distinction between Me and My Brain. And that could come from all sorts of dynamics, sandboxing of deliberative reasoning due to hostile information environments, to more easily lie in external politics, and as a result of outcomes of internal (inter-module) politics (meme wont attempt to supercede gene if meme is deluded into thinking it's already in control, so that's what gene does).
That sort of sandboxing dynamic arises inevitably from other-modelling. In order to simulate another person, you need to be able to isolate the simulation from your own background knowledge and replace it with your approximations of their own, the simulation cannot feel the brain around it. I think most peoples' conception of consciousness is like that, a simulation of what they imagine to be themselves, similarly isolated from most of the brain.
Maybe the way to transcend it is to develop a more sophisticated kind of self-model.
But that's complicated by the fact that when you're doing politics irl you need to be able to distinguish other peoples' models of you from your own model of you, so you're going to end up with an abundance of shitty models of yourself. I think people fall into a mistake of thinking that the you that your friend sees when you're talking is the actual you. They really want to believe it.
Humans sure are rough.
But after looking over this, reexamining, yeah, what causes people to talk about consciousness in these ways?
I agree. The eliminationist approach cannot explain why people talk so much about consciousness. Well, maybe it can, but the post sure doesn't try. I think your argument that consciousness is related to self-other modeling points into the right direction, but doesn't do the full work and in that sense falls short in the same way "emergence" does.
Perceiving is going on in the brain and my guess would be that the process of perceiving can be perceived too[1]. As there is already a highly predictive model of physical identity - the body - the simplest (albeit wrong) model is for the brain to identify its body and its observations of its perceptions.
Maybe the way to transcend it is to develop a more sophisticated kind of self-model.
I think that's kind of what meditation can lead to.
If AGI can become conscious (in a way that people would agree to counts), and if sufficient self-modeling can lead to no-self via meditation, then presumably AGI would also quickly master that too.
- ^
I don't know whether the brain nas some intra-brain neuronal feedback or observation-interpretation loops ("I see that I have done this action"). For LLMs, because they don't have feedback-loops internally, it could be via the context window or through observing its outputs in its training data.
I think that's kind of what meditation can lead to.
It should, right? But isn't there a very large overlap between meditators and people who mystify consciousness?
Maybe in the same way as there's also a very large overlap between people who are pursuing good financial advice and people who end up receiving bad financial advice... Some genres are majority shit, so if I characterise the genre by the average article I've encountered from it, of course I will think the genre is shit. But there's a common adverse selection process where the majority of any genre, through no fault of its own, will be shit, because shit is easier to produce, and because it doesn't work, it creates repeat customers, so building for the audience who want shit is far far more profitable.
Agree? As long as meditation practice can't systematically produce and explain the states, it's just craft and not engineering or science. But I think we will get there.
Nice post! The comments section is complex, indicating that even rationalists have a lot of trouble talking about consciousness clearly. This could be taken as evidence for what I take to be one of your central claims: the word consciousness means many things, and different things to different people.
I've been fascinated by consciousness since before starting grad school in neuroscience in 1999. Since then, I've thought a lot about consciousness, and what insight neuroscience (not the colored pictures of imaging, but detailed study of individual and groups of neurons' responses to varied situations) has to say about it.
I think it has a lot to say. There are more detailed explanations available of each of the phenomena you identify as part of the umbrella term "consciousness".
This gets at the most apt critique of this and similar approaches to denying the existence of a hard problem: "Wait! That didn't explain the part I'm interested in!". I think this is quite true, and better explanations are quite possible given what we know. I believe I have some, but I'm not sure it's worth the trouble to even try to explicate them.
Over the past 25 years, I've discussed consciousness less and less. It's so difficult as to create unproductive discussions a lot of the time, and frustrating misunderstandings and arguments a good bit of the time.
Thus, while I've wanted to write about it, there's never been a professional or personal motivating factor.
I wonder if the advent of AGI will create such a factor. If we go through a nontrivial era of parahuman AGI, as I think we will, then I think the question of whether and how they're conscious might become a consequential one, determining how we treat them.
It could also help determine how seriously we take AGI safety. If the answer to "is this proto-AGI conscious?" and the honest answer is "Yes, in some ways humans are, and some other ways humans aren't", that encourages the intuition that we should take these things seriously as a potential threat.
So, perhaps it would make sense to start that discussion now, before public debate ramps up?
If that logic doesn't strongly hold, discussing consciousness seems like a huge time-sink taking time that would be better spent trying to solve alignment as best we can while we still have a chance.
1I agree with Kaj that this is a nicely presented and readable story of your intellectual journey that teaches something on the way. I think there are a lot of parts in there that could be spun out into their own posts that would be individually more digestible to some people. My first thought was to just post this as a sequence with each chapter one post, but I think that's not the best way, really, as the arc is lost. But a sequence of sorts would still be a good idea as some topics build upon earlier ones.
One post I'd really like to be spun out is the one about pain that you have relegated to the addendum.
Sure, "everything is a cluster" or "everything is a list" is as right as "everything is emergent". But what's the actual justification for pruning that neuron? You can prune everything like that.
Great! This text by Yudkowsky has convinced me that the Philosophical Zombie thought experiment leads only to epiphenomenalism and must be avoided at all costs.
Do you mean that the original argument that uses zombies leads only to epiphenomenalism, or that if zombies were real that would mean consciousness is epiphenomenal, or what?
Sure, "everything is a cluster" or "everything is a list" is as right as "everything is emergent". But what's the actual justification for pruning that neuron? You can prune everything like that.
The justification for pruning this neuron seems to me to be that if you can explain basically everything without using a dualistic view, it is so much simpler. The two hypotheses are possible, but you want to go with the simpler hypothesis, and a world with only (physical properties) is simpler than a world with (physical properties + mental properties).
I would be curious to know what you know about my box trying to solve the meta-problem.
Do you mean that the original argument that uses zombies leads only to epiphenomenalism, or that if zombies were real that would mean consciousness is epiphenomenal, or what?
Both
Explaining everything involves explaining phenomenal consciousness, so it's literally solving the Hard Problem, as opposed to dissolving it.
The justification for pruning this neuron seems to me to be that if you can explain basically everything without using a dualistic view, it is so much simpler. The two hypotheses are possible, but you want to go with the simpler hypothesis, and a world with only (physical properties) is simpler than a world with (physical properties + mental properties).
Argument needed! You cannot go from "H1 asserts the existence of more stuff than H2" to "H1 is more complex than H2". Complexity is measured as the length of the program that implements a hypothesis, not as the # of objects created by the hypothesis.
The argument goes through for Epiphenomenalism specifically (bc you can just get rid of the code that creates mental properties) but not in general.
I would be curious to know what you know about my box trying to solve the meta-problem.
Sounds unethical. At least don't kill them afterwards.
Any conclusions would raise usual questions about how much AI's reasoning is about real things and how much it is about extrapolating human discourse. The actual implementation of this reasoning in AI could be interesting, especially given that AI would have different assumptions about its situation. But it wouldn't be necessary the same as in a human brain.
Philosophically I mostly don't see how is that different from introspecting your sensations and thoughts and writing isomorphic Python program. I guess Chalmers may agree that we have as much evidence of AIs' consciousness as of other humans', but would still ask why the thing that implements this reasoning is not a zombie?
But the most fun to think about are cases where it wouldn't apparently solve the problem: like if the reasoning was definitely generated by a simple function over relevant words, but you still couldn't find where it differs from human reasoning. Or maybe the actual implementation would be so complex, that humans couldn't comprehend it on lower level, than what we have now.
The justification for pruning this neuron seems to me to be that if you can explain basically everything without using a dualistic view, it is so much simpler.
Yeah, but can you? Your story ended on stating the meta problem, so until it's actually solved, you can't explain everything. So how did you actually check that you would be able to explain everything once it's solved? Just stating the meta problem of consciousness is like stating the meta problem of why people talk about light and calling the idea of light "a virus".
Let's put aside ethics for a minute.
"But it wouldn't be necessary the same as in a human brain."
Yes, this wouldn't be the same as the human brain; it would be like the Swiss cheese pyramid that I described in the post.
Your story ended on stating the meta problem, so until it's actually solved, you can't explain everything.
Take a look at my answer to Kaj Sotala and tell me what you think.
So maybe consciousness has always been a linguistic debate?
It has always been at least a linguistic debate , but that does not show that it as at most a linguistic debate. The thing is that the Hard Problem is already only about one specific sub-meaning of "consciousness", namely phenomenal consciousness...so you can't dissolve it just by saying "consciousness" means more than one thing, or "let's focus on the components of the problem". A problem is at least as hard as its hardest sub-problem.
And "let's focus on the components of the problem" isn't eliminativism... eliminativism the claim that there is nothing for the problem to be about.
And breaking the problem into a set of sub problems that dont contain the Hard Problem is not dissolving it. (C.f. Kaj's comments)
Don’t start by asking ‘what is consciousness’ or ‘what are qualia’; start by asking ‘what are the cognitive causes of people talking about consciousness and qualia
Which might be real things that are adequately designated by the words "consciousness"and "qualia" or real things that are not adequately designated by the words "consciousness"and "qualia", or by nothing, or...
The dictum tells you nothing.
I would be curious to know what you think about the box solving the meta-problem just before the addendum.
Do you think it is unlikely that AI would rediscover the hard problem in this setting?
I don't think it would tell you much, because it only excludes the case where the HP is a meme, not the case where it's a repeatable error.
Boxed AI tells you there is an HP.
- Could be because it is phenomenally conscious, and it has noticed there is a real HP.
- Could be because it is repeating a conceptual conclusion.
Boxed AI tells you there is not an HP.
- Could be because it is a zombie, so it can't understand what PC is.
- Could be because it has PC, but isn't subject to the erroneous thinking that causes the HP.
Note that you can't specify whether an AI is or isn't conscious, or that it's a perfect reasoner.
Note that philosophers don't agree on what constitutes a conceptual confusion.
Note that being able to trace back the causal history of an output doesn't tell you it wasn't caused by PC: one of the possible solutions to the HP is that certain kinds of physical activity, or information processing are identical to PC, so there is.not necessarily an xor between PC and physical causation. Of course, there is also a fact that human pronouncements have alone sort of causal history, and it doesn't settle much.
Note that, as things stand, the thought experiment is an intuition pump like Mary's Room, etc.
(I skipped straight to ch7, according to your advice, so I may be missing relevant parts from the previous chapters if there are any.)
I probably agree with you on the object level regarding phenomenal consciousness.
That being said, I think it's "more" than a meme. I witnessed at least two people not exposed to the scientific/philosophical literature on phenomenal consciousness reinvent/rediscover the concept on their own.
It seems to me that the first-person perspective we necessarily adopt makes inclines to ascribe to sensations/experiences some ineffable, seemingly irreducible quality. My guess is that we (re)perceive our perception as a meta-modality different from ordinary modalities like vision, hearing, etc, and that causes the illusion. It's plausible that being raised in a WEIRD culture contributes to that inclination.
A butterfly conjecture: While phenomenal consciousness is an illusion, there is something to be said about the first-person perspective being an interesting feature of some minds (sufficiently sophisticated? capable of self-reflection?). It can be viewed as a computational heuristic that makes you "vulnerable" to certain illusions or biases, such as phenomenal consciousness, but also:
- the difficulty to accept one-boxing in the Newcomb's problem
- mind-body dualism
- the naive version of free will illusion, difficulty in accepting physicalism/determinism
- (maybe) the illusion of being in control over your mind (various sources say that meditation-naive people are often surprised to discover how little control they have over their own mind when they first try meditation)
A catchy term for this line of investigation could be "computational phenomenology".
My guess is that we (re)perceive our perception as a meta-modality different from ordinary modalities like vision, hearing, etc, and that causes the illusion. It's plausible that being raised in a WEIRD culture contributes to that inclination.
This seems exceedingly unlikely. Virtually every culture has a conception of "soul" which they are confused about, and ascribe supernatural non-materialist properties to.
Yes, and the eliminationist approach doesn't explain why this is so universal and what process leads to it.
I’m confused. I know that it is like something to be me (this is in some sense the only thing I know for sure). It seems like there rules which shape the things I experience, and some of those rules can be studied (like the laws of physics). We are good enough at understanding some of these rules to predict certain systems with a high degree of accuracy, like how an asteroid will orbit a star or how electrons will be pushed through a wire by a particular voltage in a circuit. But I have no way to know or predict if it is like something to be a fish or GPT-4. I know that physical alterations to my brain seem to affect my experience, so it seems like there is a mapping from physical matter to experiences. I do not know precisely what this mapping is, and this indeed seems like a hard problem. In what sense do you disagree with my framing here?
But I have no way to know or predict if it is like something to be a fish or GPT-4
But I can predict what you say; I can predict if you are confused by the hard problem just by looking at your neural activation; I can predict word by word the following sentence that you are uttering: "The hard problem is really hard."
I would be curious to know what you think about the box solving the meta-problem just before the addendum. Do you think it is unlikely that AI would rediscover the hard problem in this setting?
this is in some sense the only thing I know for sure
You don't. All your specific experiences are imprecise approximations: you can't be sure what exact color you saw for how many nanoseconds, you can't be sure all your brain except small part implementing only current thought haven't evaporated microsecond ago. So you can have imprecise models of a fish brain the same way you have imprecise models of your brain - your awareness of your brain is casually connected to your brain the same way your thoughts can be casually connected to a fish brain. You just can't be fully fish.
So, is your conclusion ("the place where one stops writing") that there is an unsolved hard problem, there is a solved hard problem, or there is no hard problem?
I don't know, it depends on your definition of "unsolved" and "solved", but I would lean towards "there is a solved hard problem" because the problem was hard, it took me a lot of time (i.e. the meme of the hard problem existed in my head), and my post finally dissolved the question.
I think your 100 billion people holding thousands of hands each are definitely conscious. I also think the United States and in fact nearly every nationstate are probably conscious as well. Also, my Linux system may be conscious.
I believe consciousness is, at its core, a very simple system: something closer to the differentiation operator than to a person. We merely think that it is a complicated big thing because we confuse the mechanism with the contents - a lot of complicated systems in the brain exchange data using consciousness in various formats, including our ego. However, I believe just consciousness - the minimal procedure in itself - is simply not actually very mysterious or complicated as far as algorithms go.
A meme, in text form:
"Textbooks from the Future"
A time traveller handing a professor a book, simply labeled "Consciousness". The man is clearly elated, as the book is very thin. Behind the time traveller's back is a trolley carrying a very thick book labelled "Cognitive algorithms that use consciousness, vol. 1."
I guess Daniel Dennett was right in the end, ha ha. I liked how his idea of there being no problem was dismissed so quickly in the text. I like his approach of degrees of freedom and very evolutionary view on the whole mentality of humans and other intelligence.
It seems to me that you've had a long intellectual journey and, judging from your comments on AI and consciousness, now you would endorse the maxim "Once you've explained everything that happens, you've explained everything." Is that right?
If so, I really suggest going back to Dennett — I think now you will be nodding along enthusiastically. In my opinion he doesn't dodge or ignore the hard problem, but gives very good explanations of why it is an artifact. If you haven't read "Quining Qualia", I suggest you go there first — a very approachable essay on why qualia do not exist.
Completely wrong conclusion - but can you also explain how this is supposed to relate to Yann LeCun's views on AI safety?
Frontpage comment guidelines:
- Aim to explain, not persuade
- Try to offer concrete models and predictions
- If you disagree, try getting curious about what your partner is thinking
- Don't be afraid to say 'oops' and change your mind
I would like to defer any debate over your conclusion for a moment, because that debate is not new. But this is:
I think one of the main differences in worldview between LeCun and me is that he is deeply confused about notions like what is true "understanding," what is "situational awareness," and what is "reasoning," and this might be a catastrophic error.
This is the first time I've heard anyone say that LeCun's rosy views of AI safety stem from his philosophy of mind! Can you say more?
I'm not saying that LeCun's rosy views on AI safety stem solely from his philosophy of mind, but yes, I suspect there is something there.
It seems to me that when he says things like "LLMs don't display true understanding", "or true reasoning", as if there's some secret sauce to all this that he thinks can only appear in his Jepa architecture or whatever, it seems to me that this is very similar to the same linguistic problems I've observed for consciousness.
Surely, if you will discuss with him, he will say things like "No, this is not just a linguistic debate, LLMs cannot reason at all, my cat reasons better": This surely indicates a linguistic debate.
It seems to me that LeCunis is basically an essentialist of his Jepa architecture, as the main criterion for a neural network to exhibit "reasoning".
LeCun's algorithm is something like: "Jepa + Not LLM -> Reasoning".
My algorithm is more something like: "chain-of-thought + can solve complex problem + many other things -> reasoning".
This is very similar to the story I tell for consciousness in the Car Circuit section here.
OK, maybe I understand. If I put it in my own words: You think "consciousness" is just a word denoting a somewhat arbitrary conjunction of cognitive abilities, rather than a distinctive actual thing which people are right or wrong about in varying degrees, and that the hard problem of consciousness results from reifying this conjunction. And you suspect that LeCun in his own thinking e.g. denies that LLMs can reason, because he has added unnecessary extra conditions to his personal definition of "reasoning".
Regarding LeCun: It strikes me that his best-known argument about the capabilities of LLMs rests on a mathematical claim, that in pure autoregression, the probability of error necessarily grows. He directly acknowledges that if you add chain of thought, it can ameliorate the problem... In his JEPA paper, he discusses what reasoning is, just a little bit. In Kahneman's language, he calls it a system-2 process, and characterizes it as "simulation plus optimization".
Regarding your path to eliminativism: I am reminded of my discussion with Carl Feynman last year. I assume you both have subjective experience that is made of qualia from top to bottom, but also have habits of thought that keep you from seeing this as ontologically problematic. In his case, the sense of a problem just doesn't arise and he has to speculate as to why other people feel it; in your case, you felt the problem, until you decided that an AI civilization might spontaneously develop a spurious concept of phenomenal consciousness.
As for me, I see the problem and I don't feel a need to un-see it. Physical theory doesn't contain (e.g.) phenomenal color; reality does; therefore we need a broader theory. The truth is likely to sound strange, e.g. there's a lattice of natural qubits in the cortex, the Cartesian theater is how the corresponding Hilbert space feels from the inside, and decohered (classical) computation is unconscious and functional only.
1in your case, you felt the problem, until you decided that an AI civilization might spontaneously develop a spurious concept of phenomenal consciousness.
This is the best summary of the post currently