Against functionalism: a self dialogue
4James Diacoumis
17[anonymous]
3James Diacoumis
4[anonymous]
2Algon
1James Diacoumis
4[anonymous]
4[anonymous]
6Algon
New Comment
I found this post pretty helpful to crystallise two distinct views that often get conflated. I’ll call them abstract functionalism and physical functionalism. The key confusion comes from treating these as the same view.
When we talk about a function it can be instantiated in two ways: abstractly and physically. On this view there’s a meaningful difference between an abstract instantiation of a function, such as a disembodied truth table representing a NAND gate and a physical instantiation of a NAND gate e.g. on a circuit board with wires and voltages etc..
When S argues:
The causal graph of a bat hitting a ball might describe momentum and position, but if you re-create that graph elsewhere (e.g. on a computer or some scaled) it won't have that momentum or velocity
They’re right that abstract function leaves out some critical physical properties. A simulation of momentum transfer doesn’t actually transfer momentum. But this doesn’t defeat functionalism it just shows that abstract instantiation of the function is not enough.
For example, consider a steel wing and a birds wing generating lift. The steel wing has vastly different kinetic energy requirements but the aerodynamics still works because steel can support the function. Contrast this with combustion - steel can’t burn like wood because it lacks the right chemical energy profile.
When A asks:
Do you claim that, if I started replacing neurons in your brain with stuff that is functionally the same, wrt. the causal graph of consciousness, you'd feel no difference? You'd still be conscious in the same way?
They’re appealing to the intuition that physically instantiated functional replicas of neurons would preserve consciousness.
The distinction matters because people often use the “simulations lack physical properties” argument to dismiss abstract functionalism and then tie themselves in knots trying to understand whether a physically embodied AI robot system could be conscious when they haven’t defeated physical functionalism.
Separately from my other comment, I have responses here as well.
When we talk about a function it can be instantiated in two ways: abstractly and physically. On this view there’s a meaningful difference between an abstract instantiation of a function, such as a disembodied truth table representing a NAND gate and a physical instantiation of a NAND gate e.g. on a circuit board with wires and voltages etc..
Indeed. As I wrote about here:
I think it would be non-physicalist if (to slightly modify the analogy, for illustrative purposes) you say that a computer program I run on my laptop can be identified with the Python code it implements, because it is not actually what happens.
We can see this as a result of stuff like single-event upsets, i.e., for example, situations in which stray cosmic rays modify the bits in a transistor in the physical entity that runs the code (i.e., the laptop) in such a manner that it fundamentally changes the output of the program. So the running of the program (instantiated and embedded in the real, physical world just like a human is) works not on the basis of the lossy model that only takes into account the "software" part, but rather on the "hardware" itself.
You can of course expand the idea of "computation" to say that, actually, it takes into account the stray cosmic rays as well, and in fact it takes into account everything that can affect the output, at which point "computation" stops being a subset of "what happens" and becomes the entirety of it. So if you want to say that the computation necessarily involves the entirety of what is physically there, then I believe I agree, at which point this is no longer the computationalist thesis argued for by Rob, Ruben etc (for example, the corolaries about WBE preserving identity when only an augmented part of the brain's connectome is scanned no longer hold).
The basic point, which I think I come back to over and over again,[1] is that the tails come apart when we get down to the nuts and bolts of the fundamental nature of reality if we continue to talk about important topics imprecisely.[2] In a casual conversation with an SWE friend, it's fine and even desirable to conflate the two instantiations of a function. Not because it's correct, mind you, but because the way it's incorrect is likely irrelevant to the topic at hand[3] and being super careful about it wastes time and sounds pedantic. But when you zoom in super deeply and try to get at the core of what makes existence tick, not only are we in an area where we don't have nearly as much knowledge to justify certainty of this kind, but also any errors we make in our use or delineation of concepts get magnified and lead us down the wrong tracks. We have to bind ourselves tight to reality, which is very hard to do in such a confusing domain.
But this doesn’t defeat functionalism it just shows that abstract instantiation of the function is not enough.
Sure sounds like a defeat of (that particular example of) functionalism to me! You need more than just the lossy compression you're using. You need to pay attention to the map-territory distinction and abstain from elevating a particular theory to undeserved heights. And maybe, just maybe, you need to consider what the physical substrate actually does instead of writing down imperfect abstract mathematical approximations of it.
For example, consider a steel wing and a birds wing generating lift. The steel wing has vastly different kinetic energy requirements but the aerodynamics still works because steel can support the function. Contrast this with combustion - steel can’t burn like wood because it lacks the right chemical energy profile.
Steel wings and bird wings are similar in some ways. They also differ in some ways, as you pointed out. The more differences you recognize, the more you constrain the possibility space for what kinds of mathematical structures you can build from the ground up to ensure all the differences (and similarities) are faithfully represented.[4] What's to say that once you consider all the differences,[5] you're left with nothing but... well, everything? The entire physical instantiation of what's going on?
The distinction matters because people often use the “simulations lack physical properties” argument to dismiss abstract functionalism and then tie themselves in knots trying to understand whether a physically embodied AI robot system could be conscious when they haven’t defeated physical functionalism.
As a general matter, I'm not entirely certain what people are truly pointing to in the territory when they say "consciousness."[6] I don't know what its True Name is. I'm worried it doesn't really refer to much once we dissolve our confusions. Whenever I think of this topic, I'm always reminded of this absolutely excellent lc post:
It is both absurd, and intolerably infuriating, just how many people on this forum think it's acceptable to claim they have figured out how qualia/consciousness works, and also not explain how one would go about making my laptop experience an emotion like 'nostalgia', or present their framework for enumerating the set of all possible qualitative experiences. When it comes to this particular subject, rationalists are like crackpot physicists with a pet theory of everything, except rationalists go "Huh? Gravity?" when you ask them to explain how their theory predicts gravity, and then start arguing with you about gravity needing to be something explained by a theory of everything. You people make me want to punch my drywall sometimes.
For the record: the purpose of having a "theory of consciousness" is so it can tell us which blobs of matter feel particular things under which specific circumstances, and teach others how to make new blobs of matter that feel particular things. Down to the level of having a field of AI anaesthesiology. If your theory of consciousness does not do this, perhaps because the sum total of your brilliant insights are "systems feel 'things' when they're, y'know, smart, and have goals. Like humans!", then you have embarassingly missed the mark.
I suppose my interest in lc's questions means I do care about the stuff @Algon mentioned, at least to some extent.
- ^
But probably haven't really spelled out in so many words yet.
- ^
Or by reifying concepts that don't carve reality at the joints.
- ^
And we know this because we have a tremendous amount of background knowledge amassed by experts over the decades that give us a detailed, mechanistic explanation of how unlikely this is to matter.
- ^
A greater complexity, more epicycles, if you will.
- ^
Which is relevant here and not in other discussions about other topics because, as I explained above, this is a qualitatively different domain.
- ^
And I claim they are equally uncertain (or, in most cases, should be equally or even less certain than me when it comes to this).
1As a clarification, I'm working with the following map:
- Abstract functionalism (or computational functionalism) - the idea that consciousness is equivalent to computations or abstractly instantiated functions.
- Physical functionalism (or causal-role functionalism) - the idea that consciousness is equivalent to physically instantiated functions at a relevant level of abstraction.
I agree with everything you've written against 1) in this comment and the other comment so will focus on defending 2).
If I understand the crux of your challenge to 2), you're essentially saying that once we admit physical instantiation matters (e.g. cosmic rays can affect computations, steel vs birds wings have different energy requirements) then we're on a slippery slope because each physical difference we admit further constrains what counts as the "same function" until we're potentially only left with the exact physical system itself. Is this an accurate gloss of your challenge?
Assuming it is, I have a couple of responses:
I actually agree with this to an extent. There will always be some important physical differences between states unless they're literally physically identical at a token level. The important thing is to figure out which level of abstraction is relevant for the particular "thing" we're trying to pin down. We shouldn't commit ourselves to insisting that systems which are not physically identical can't be grouped in a meaningful way.
On my view, we can't need an exact physical duplicate to reflect presence/absence of consciousness because consciousness is so remarkably robust. The presence of consciousness persists over multiple time-steps in which all manner of noise, thermal fluctuations and neural plasticity occur. What changes is the content/character of consciousness - but consciousness persists because of robust higher-level patterns not because of exact microphysical configurations.
And maybe, just maybe, you need to consider what the physical substrate actually does instead of writing down imperfect abstract mathematical approximations of it.
Again, I agree that not every physical substrate can support every function (I gave the example of combustion not being supported in steel above.) If the physical substrate prevents certain causal relations from occurring then this is a perfectly valid reason for it not to support consciousness. For example, I could imagine that it's physically impossible to build embodied robot AI systems which pass behavioural tests for consciousness because the energy constraints don't permit it or whatever. My point is that in the event where such a system is physically possible then it is conscious.
To determine if we actually converge or if there's a fundamental difference in our views: Would you agree that if it's possible in principle to build a silicon replica of a brain at whatever the relevant level of abstraction for consciousness is (whether coarse-grained functional level, neuron-level, sub-neuron level or whatever) then the silicon replica would actually be conscious?
If you agree here, or if you insist that such a replica might not be physically possible to build then I think our views converge. If you disagree then I think we have a fundamental difference about what constitutes consciousness.
Would you agree that if it's possible in principle to build a silicon replica of a brain at whatever the relevant level of abstraction for consciousness is (whether coarse-grained functional level, neuron-level, sub-neuron level or whatever) then the silicon replica would actually be conscious?
Yeah, I think this is pretty likely.[1]
I will say I do believe, in general, that we simply need a much better understanding of what "consciousness" means before we can reason more precisely about these topics. Certain ontologies can assign short encodings to concepts that are either ultimately confused or at the very least don't carve reality at the joints.
We typically generalize from one example when it comes to consciousness and subjectivity: "we're conscious, so therefore there must be some natural concept consciousness refers to" is how the argument goes. And we reject solipsism because we look at other human beings and notice that they act similarly to us and seem to possess the same internal structure as us, so we think we can safely say that they must also have an "internal life" of subjectivity, just like us. That's all fine and good. But when we move outside of that narrow, familiar domain and try to reason about stuff our intuition was not built for, that's when the tails come apart and stuff can get very weird.
But overall, I don't think we disagree about too much here. I wouldn't talk about this topic in the terms you chose, and perhaps this does reflect some delta between us, but it's probably not a major point of disagreement.
- ^
As to the additional question of identity, namely whether that replica is the same consciousness as that which it's meant to replicate... I'd still say no. But that doesn't seem to be what you're focused on here.
I found this post pretty helpful to crystallise two distinct views that often get conflated.
: ) I'm glad you found this helpful. I was unsure whether it would be clear to others.
I’ll call them abstract functionalism and physical functionalism
I wonder what the standard terminology is. Also, I think you could easily swap out "abstract functionalism" for "computationalism" in this dialogue, with a few minor tweaks to wording, and it would still work. Indeed, that's how I initially wrote it. So using your wording, I'd say this dialogue's really aiming to crystallize the difference between physical functionalism and other philosophies of mind.
Changed "some scaled" to "or some re-scaled version of the system". Thanks for making that error salient!
The distinction matters because people often use the “simulations lack physical properties” argument to dismiss abstract functionalism and then tie themselves in knots trying to understand whether a physically embodied AI robot system could be conscious when they haven’t defeated physical functionalism.
Hadn't thought of this, though I think the physical functionalist could go either way on whether a physically embodied robot wouldn't be conscious.
As an aside, I think looking at how neurons actually work would probably resolve the disagreement between my inner A and S. Like, I do think that if we knew that the brain's functions don't depend on sub-neuron movements, then the neuron-replacement argument would just work. But since S is meant to simulate @sunwillrise, and they are a high perplexity individual, I may well be wrong on whether they'd find that convincing.
1I think the physical functionalist could go either way on whether a physically embodied robot wouldn't be conscious.
Just clarifying this. A physical functionalist could coherently maintain that it’s not possible to build an embodied AI robot because physics doesn’t allow it. Similar to how a wooden rod can burn but a steel rod can’t because of the physics. But assuming that it’s physically possible to build an embodied AI system which passes behavioural tests of consciousness e.g. self-recognition, cross-modal binding, flexible problem solving etc.. then the physical functionalist would maintain that the system is conscious.
I think looking at how neurons actually work would probably resolve the disagreement between my inner A and S. Like, I do think that if we knew that the brain's functions don't depend on sub-neuron movements, then the neuron-replacement argument would just work
Out of interest, do you or @sunwillrise have any arguments or intuitions that the presence or absence of consciousness turns on sub-neuronal dynamics?
Consciousness appears across radically different neural architectures; octopuses with distributed neural processing in their arms, birds with a nucleated brain structure called the pallium which differs from the human cortex but has similar functional structure, even bumblebees are thought to possess some form of consciousness with far fewer neuron counts than humans. These examples exhibit coarse-grained functional similarities with the human brain - but differ substantially at the level of individual neurons.
If sub-neuronal dynamics determined presence or absence of consciousness we’d expect minor perturbations to erase it. Instead we’re able to lesion large brain regions whilst maintaining consciousness. You also preserve consciousness when small sub-neuronal changes are applied to every neuron such as when someone takes drugs like alcohol or caffeine. Fever also alters reaction rates and dynamics in every neuron across the brain. This robustness indicates that presence or absence of consciousness turns on coarse-grained functional dynamics rather than sub-neuronal dynamics.
Out of interest, do you or @sunwillrise have any arguments or intuitions that the presence or absence of consciousness turns on sub-neuronal dynamics?
I will point to what I (and others) have commented on or posted about before, and I shall hope that's an adequate answer to your question.
I have written:
As TAG has written a number of times, the computationalist thesis seems not to have been convincingly (or even concretely) argued for in any LessWrong post or sequence (including Eliezer's Sequences). What has been argued for, over and over again, is physicalism, and then more and more rejections of dualist conceptions of souls.
That's perfectly fine, but "souls don't exist and thus consciousness and identity must function on top of a physical substrate" is very different from "the identity of a being is given by the abstract classical computation performed by a particular (and reified) subset of the brain's electronic circuit," and the latter has never been given compelling explanations or evidence. [1] This is despite the fact that the particular conclusions that have become part of the ethos of LW about stuff like brain emulation, cryonics etc are necessarily reliant on the latter, not the former.
As a general matter, accepting physicalism as correct would naturally lead one to the conclusion that what runs on top of the physical substrate works on the basis of... what is physically there (which, to the best of our current understanding, can be represented through Quantum Mechanical probability amplitudes), not what conclusions you draw from a mathematical model that abstracts away quantum randomness in favor of a classical picture, the entire brain structure in favor of (a slightly augmented version of) its connectome, and the entire chemical make-up of it in favor of its electrical connections. As I have mentioned, that is a mere model that represents a very lossy compression of what is going on; it is not the same as the real thing, and conflating the two is an error that has been going on here for far too long. Of course, it very well might be the case that Rob and the computationalists are right about these issues, but the explanation up to now should make it clear why it is on them to provide evidence for their conclusion.
I have also written:
More specifically, is a real-world being actually the same as the abstract computation its mind embodies? Rejections of souls and dualism, alongside arguments for physicalism, do not prove the computationalist thesis to be correct, as physicalism-without-computationalism is not only possible but also (as the very name implies) a priori far more faithful to the standard physicalist worldview.
[...]
The feedback loops implicit in the structure of the brain cause reward and punishment signals to "release chemicals that induce the brain to rearrange itself" in a manner closely analogous to and clearly reminiscent of a continuous and (until death) never-ending micro-scale brain surgery. To be sure, barring serious brain trauma, these are typically small-scale changes, but they nevertheless fundamentally modify the connections in the brain and thus the computation it would produce in something like an emulated state (as a straightforward corollary, how would an em that does not "update" its brain chemistry the same way that a biological being does be "human" in any decision-relevant way?). We can think about a continuous personal identity through the lens of mutual information about memories, personalities etc, but our current understanding of these topics is vastly incomplete and inadequate, and in any case the naive (yet very widespread, even on LW) interpretation of "the utility function is not up for grabs" as meaning that terminal values cannot be changed (or even make sense as a coherent concept) seems totally wrong.
[...]
The way communities make progress on philosophical matters is by assuming that certain answers are correct and then moving on. After all, you can't ever get to the higher levels that require a solid foundation if you aren't allowed to build such a foundation in the first place. But I worry, for reasons that have been stated before, that the vast majority of the discourse by "lay lesswrongers" (and, frankly, even far more experienced members of the community working directly on alignment research; as a sample illustration, see a foundational report's failure to internalize the lesson of "Reward is not the optimization target") is based on conclusions reached through informal and non-rigorous intuitions that lack the feedback loops necessary to ground themselves to reality because they do not do enough "homework problems" to dispel misconceptions and lingering confusions about complex and counterintuitive matters.
TAG has written:
Naturalism and reductionism are not sufficient to rigourously prove either form of computationalism -- that performing a certain class of computations is sufficient to be conscious in general, or that performing a specific one is sufficient to be a particular conscious individual.
This has been going on for years: most rationalists believe in computationalism, none have a really good reason to.
Arguing down Cartesian dualism (the thing rationalists always do) doesn't increase the probability of computationalism, because there are further possibilities , including physicalism-without-computationalism (the one rationalists keep overlooking) , and scepticism about consciousness/identity.
One can of course adopt a belief in computationalism, or something else, in the basis of intuitions or probabilities. But then one is very much in the ream of Modest Epistemology, and needs to behave accordingly.
"My issue is not with your conclusion, it’s precisely with your absolute certainty, which imo you support with cyclical argumentation based on weak premises".
Andesoldes has written:
You're missing the bigger picture and pattern-matching in the wrong direction. I am not saying the above because I have a need to preserve my "soul" due to misguided intuitions. On the contrary, the reason for my disagreement is that I believe you are not staring into the abyss of physicalism hard enough. When I said I'm agnostic in my previous comment, I said it because physics and empiricism lead me to consider reality as more "unfamiliar" than you do (assuming that my model of your beliefs is accurate). From my perspective, your post and your conclusions are written with an unwarranted degree of certainty, because imo your conception of physics and physicalism is too limited. Your post makes it seem like your conclusions are obvious because "physics" makes them the only option, but they are actually a product of implicit and unacknowledged philosophical assumptions, which (imo) you inherited from intuitions based on classical physics. By this I mean the following:
It seems to me that when you think about physics, you are modeling reality (I intentionally avoid the word "universe" because it evokes specific mental imagery) as a "scene" with "things" in it. You mentally take the vantage point of a disembodied "observer/narrator/third person" observing the "things" (atoms, radiation etc) moving, interacting according to specific rules and coming together to create forms. However, you have to keep in mind that this conception of reality as a classical "scene" that is "out there" is first and foremost a model, one that is formed from your experiences obtained by interacting specifically with classical objects (biliard balls, chairs, water waves etc). You can extrapolate from this model and say that reality truly is like that, but the map is not the territory, so you at least have to keep track of this philosophical assumption. And it is an assumption, because "physics" doesn't force you to conclude such a thing. Seen through a cautious, empirical lens, physics is a set of rules that allows you to predict experiences. This set of rules is produced exclusively by distilling and extrapolating from first-person experiences. It could be (and it probably is) the case that reality is ontologically far weirder than we can conceive, but that it still leads to the observed first-person experiences. In this case, physics works fine to predict said experiences, and it also works as an approximation of reality, but this doesn't automatically mean that our (merely human) conceptual models are reality. So, if we want to be epistemically careful, we shouldn't think "An apple is falling" but instead "I am having the experience of seeing an apple fall", and we can add extra philosophical assumptions afterwards. This may seem like I am philosophizing too much and being too strict, but it is extremely important to properly acknowledge subjective experience as the basis for our mental models, including that of the observer-independent world of classical physics. This is why the hard problem of consciousness is called "hard". And if you think that it should "obviously" be the other way around, meaning that this "scene" mental model is more fundamental than your subjective experiences, maybe you should reflect on why you developed this intuition in the first place. (It may be through extrapolating too much from your (first-person, subjective) experiences with objects that seemingly possess intrinsic, observer-independent properties, like the classical objects of everyday life.)
At this point it should be clearer why I am disagreeing with your post. Consciousness may be classical, it may be quantum, it may be something else. I have no issue with not having a soul and I don't object to the idea of a bunch of gears and levers instantiating my consciousness merely because I find it a priori "preposterous" or "absurd" (though it is not a strong point of your theory). My issue is not with your conclusion, it's precisely with your absolute certainty, which imo you support with cyclical argumentation based on weak premises. And I find it confusing that your post is receiving so much positive attention on a forum where epistemic hygiene is supposedly of paramount importance.
