The last post, "Putting it all together" served to close the loop and complete the framework. From here, we just have to tie up some loose ends and demarcate the limitations of what can be done both physiologically and practically. This post discusses the former.
Common perspective is that unreasonable people aren't updating on evidence when we know they should. Irrational fear is about our own brains somehow not doing what we know they should. Pain is about nociceptors firing. Swelling is just a thing that happens. Depression and bipolar disorder are about chemical imbalances. Alzheimer's is about physical brain degradation. Phineas Gage's change in temperament was because he got a railroad stake through the brain, and gunshots to the head kill by physical damage -- not by convincing people it's no longer worth beating their hearts, breathing, and living life.
By exploring the framing of "disagreements" and the fundamental requirements for navigating them successfully assuming the underlying neurology is updating properly on new information, we've noticed that common perspective overstates the physical limitations and understates the degree to which we just don't know what's right. Yet clearly by the time we get to the end of that list, "physical malfunction" becomes a significant part of the story.
Now that we've seen how to navigate disagreements when the underlying hardware works well enough to be abstracted away, we're equipped to notice when it doesn't, and explore how that changes the analysis.
Let's return to my temperature controller, to start with.
Back in "Expectation = intention = setpoint" I quipped that even temperature controllers can have insecurity issues. My heat treat oven temperature controller self censors if it ever tries to expect to put out more than about 40% of maximum power. It embodies the expectation that the outcome won't be better if it engages with the full extent of prediction error, and limits its ability to address that prediction error as a result.
What if we wanted to teach it that it's safe to be more secure? What if I changed the elements out for elements that can handle the full output? I can and do change the code to update such expectations, but there's nothing you can do from the outside to fix this. If you bypass the controller and force full power, it won't notice. Even if it had a current sensor and was closing the loop, there's no code there detecting whether it's still working, let alone code reasoning from here to what kinds of limits are likely safe. It's just hard coded, and if you provide it valid evidence that what it believes to be dangerous is in fact safe, it won't respond.[1]
A person might give you the cold shoulder, or put on a poker face. Or they might let their eyes glaze over when your ramblings have no relevance to them. The temperature controller won't "ignore" you, it won't sense you. If you bring out a sledgehammer and threaten to destroy it, the temperature controller won't so much as blink. It will keep doing exactly as it was doing before.
Do you ever see such clear failure to respond to obviously relevant stimuli in people?
Ever see a person fail to respond to a sledge hammer raised over their head? That's one way to get a response out of someone who is trying to ignore you.
Or fail to respond in any way to getting blatantly laughed at? Not to stand their ground, or get defensive, or to try to ignore you, but to be in a position to actually care, and somehow not notice? I have. But the case that comes to mind involves talking with someone who was quite drunk... and on LSD... and, it turns out, asleep. So I think it's fair to hypothesize that his hardware wasn't very functional at processing information at that time. And, now that I think of it, he probably didn't care too much either.
When someone's brain is doing something dumb, it's often not easy to get them to agree. Or to hear you out. And people will often miss, or misinterpret, or choose to ignore signals that they're wrong, at first. But usually, if you have some credible reason to think their brain is doing the wrong thing, you can get them to notice. Their response might be a defensive flinch, or a conspicuous lack of active response where there otherwise would be, but even that is responding to the stimulus with a change in behavior. It's not often that you can escalate things all the way and have people act exactly the same as if they don't see the evidence, the way my temperature controller acts like it doesn't see the evidence.
Usually, what happens when you tell people "You're definitely wrong" is that they respond to that information in some way -- and either they agree or you have a disagreement to sort out. Generally people are functionally intelligent enough to notice the disagreements, but that doesn't necessarily mean their hardware is functional enough to allow resolution of the disagreement.
What would it have looked like if my friend was right that her fear was irrational? If her fear was actually immune to evidence due to a brain tumor or something, how would we notice? What are the signs that a disagreement is actually unresolvable due to hardware limitations, rather than simply an inability to provide the appropriate evidence?
Take a moment to think about it for yourself.
And when you're ready, we can explore some actual examples.
Like the time my friend in high school came in to class and started telling me about how his head hurts, and his arm hurts, because he had been jumping on some cardboard boxes outside and fell off hitting his head and his arm. A minute later he started telling me again about how his head hurts, and his arm hurts, but this time he didn't know why. I told him he fell when jumping on some boxes, and he wouldn't believe me. He thought I was messing with him. That's a disagreement.
If his amnesia were purely the result of software, like "hypnotic amnesia" is, then in order to maintain the amnesia he would have to find reasons to not look. That's not what happened though. He didn't protest that "I can't remember! I have amnesia, remember!? You can't expect me to remember!". He didn't insist that he's "trying", while directing his attention towards me, or his backpack, or any other place that's unlikely to contain the memory. He didn't look like "This makes sense, I don't expect to remember what I did 5 minutes ago".
When I told him that I was serious, he looked confused. Like, "If that's true -- and I believe he's probably right -- then I would expect to remember that. And that expectation is not being realized". I could see him... well, I couldn't quite see him looking for the memory, but I could see him not focusing on why he didn't have to look, and not focusing on anything else. He sure looked like he was actually looking, and drawing a blank. It's like when you're fumbling for a word that's on the tip of your tongue, saying "Can I borrow your... ugh... what's that word. Hm................. OH! 'Calculator!'". He was visibly looking, only the memory never came. In other words, his behavior was better explained by a genuine hardware malfunction.
This is unsurprising given that he had just bonked his head on the concrete, but knowledge of this blow to his head isn't necessary in order to notice that this doesn't look like software. He wasn't just saying that he should be able to remember, he was actually aiming his attention in the directions needed to retrieve the memory and it wasn't working.
We were able to agree that he had amnesia due to a blow to the head, and that I'd walk him to the nurses office, but when it came to the memory retrieval his neurology just wasn't doing its job. There was a legitimate physical malfunction, and a disagreement resulting from that. Resolving the disagreement was necessary in order to do the appropriate thing for the context, but that doesn't automatically fix the physical damage or bring back the memory.
The chronic pain example also has misfiring neurology. His nerves weren't actually sensing "everything", but that's how it felt because the nerves were physically damaged and misfiring constantly. The conversation I had with him didn't fix his nerves overnight so that instead of "feels like everything" it felt only like what was there. It just helped him contextualize what these nerve signals meant. His nerves weren't accurately mapping reality, but his brain was able to map that. Instead of expecting his nerves to map reality, and expecting this expectation to be realized (thereby driving him crazy when nothing was going as expected) he could notice that this expectation was going to fail -- at least for a time -- and make sense of what to do now that these nerves which were formerly part of his roughly coherent system were now more of an "external noise". The overnight benefit of that conversation was not in fixing the reality of his damaged neurology, but in helping him understand how best to navigate the new reality that he found himself in -- partially malfunctioning neurology and all.
Returning now to my friend and her fear of heights, can you see what it would look like if her problem were indeed hardware?
If you want to take another moment to test your understanding, do so now.
...
...
Got your answer?
Having a fear that doesn't respond to evidence is like having nerves that don't calm down once you realize that you're not actually sensing "everything". If the nerves don't respond to reality, you stop responding to them. You stop acting like the fear means anything, once you realize that it doesn't. Okay, I'm afraid -- so what. It's not dangerous. It becomes funny that you're afraid, and the fear stops being bothersome. You just route around it, and depending on exactly where the disconnect is, it might even stop feeling like "fear" and start feeling more like "inappropriate excitement".
This is also what it feels like with functional neurology, by the way. It's just that with functional neurology it's transient, as the object level beliefs update to match the evidence. For a month or so after a particularly bad experience I would have bouts of seemingly content-less anxiety come up for seemingly no reason. But it didn't bother me because I knew the bad was over, so it was just "Ooh, neat. Anyway..". It was just a thing my body was doing, and it didn't mean anything. Uncoincidentally, it stopped happening.
Notice that whenever you have some "irrational" behavior that isn't going away, it's not funny? Notice how if you try to laugh, it just brings up other feelings which conflict with that laughter? Feelings about how you shouldn't be feeling this way... as if you could feel differently... if only you were to do something about it that you're not doing"? It's theoretically possible to laugh at your own responses and route around it while they stick around -- if your hardware isn't working -- but I've never seen it happen.
Okay, moving on to bigger scarier problems. What about depression and chemical imbalances?
"Chemical imbalance" alone doesn't say much. We're built on neurochemistry, so it's no surprise there are neurochemical correlates; it'd be a surprise if there weren't. The question isn't whether there are neurochemical correlates, the question is whether those neurochemical markers are tracking the evidence or not -- and that's a harder question to answer.
Sometimes people are more depressed seasonally, and bright lights help. Sometimes fixing nutrition elevates mood. That seems to suggest that the chemistry is tracking things other than "rational evidence of how good life is". I don't think that's the full story.
Say you feel bad all the time due to poor nutrition. You're unhappy, and find it hard to get motivated to do anything. Well, what do you expect? If you're in poor shape and don't have energy to do much, of course you can't find motivation to do much -- you don't have resources to do much, so that what tracking reality looks like. If you're unhappy, is it just because you want to have more energy to run around and do stuff? That sure sounds reasonable, and "let go, and be content to sit around with no energy" doesn't sound like a great solution, when compared to "get your nutritional problems figured out". There's nothing blank about "Why not be happy?", the way my friend's mind was blank when trying to remember how he hurt his head. It's just this actually sucks. The problem is real, and the beliefs are doing their job by pointing at it.
Continuing the same pattern, this "depression" is a disagreement coming from a real limitation. That's important to notice, because if you know that your depression stems from malnutrition and a real lack of physiological resources to do things with, then you know to see a nutritionist rather than a psychiatrist. And clear thinking keeps us from spiraling into unsupported catastrophizing, so things aren't worse than they have to be given our real limitations.
While that's clearly a part of it, and I've seen it, I don't get the sense that most depression is even this physiological. Certainly my own experience with it wasn't. Everyone likes to say "You can't just decide to not be depressed!", but it's factually untrue.
When I was in middle school, my mom noticed that I had seemed depressed, and asked if I was. Like my cousin with the fire poker, I noticed that my immediate answer wasn't what I wanted to stick with, and by the time I opened my mouth to say "Nope", it had become true. In that moment, I decided to not be depressed, and it worked.[2]
I don't talk about it much because normally it'd just come off like "I could do it so you should too!" which isn't actually correct, and it would take many words to convey the correct meaning. And because "Good for you. What do you want, a cookie?" -- and I don't like cookies. But after spending tens of thousands of words explaining exactly why "just deciding" is so hard yet actually possible, it's worth pointing out that yes, this applies to depression too. There's just a selection bias in that the cases that are that easy to solve don't tend to gather attention in the first place, and people aren't exactly incentivized to share their stories.[3][4]
Similarly, "bipolar" looks a lot like what you'd expect when someone oscillates from flinching from the good to flinching from the bad, changing strategies for how to avoid prediction error out of insecurity as each direction becomes unsustainable. Most people would think to lift people up when they're depressed and "bring people down to earth" when manic, but if you know anything about control theory you'll know that this doesn't provide damping. If anything, you might be a bit slow to notice phase shifts, and end up contributing to the instability. Instead, if you look for the flinches from reality and orient to truth, you'll end up leading the mood, which provides damping. This means tearing people down even at their lowest, as they start to ignore the bad instead of the good. This also means supporting mania at their peak when the arguments starting to bring them down are flinching from reality. I only have brief experience with bipolar, because the only person I know who had bipolar disorder stopped having it after one cycle of paying attention to call her out on her nonsense at the relevant phases. She's quite happy and stable now, without mood stabilizers, so it looks like at least in her case it really was as simple as that.
Let's try a harder one. What about schizophrenia?
I'm not an expert on schizophrenia. I haven't even met anyone suffering from it. I've just skimmed through some papers on how it looks from a predictive coding perspective. Normally, this is where I'd say "Therefore take my perspective with a grain of salt", except that my whole thesis is "Don't do things you can't actually expect to be worth the risk, after taking into account whether your own perspective is worth anything and whether it's safe to even consider whether your perspective is worth anything". So instead of telling you to take what I say with a grain of salt, I'm just going to double down: don't do things you can't actually expect to be worth the risk, after taking into account whether your own perspective is worth anything and whether it's safe to even consider whether your perspective is worth anything.
With that said, from my relatively uneducated perspective on schizophrenia, it looks like "Confidently predicting that everything is unpredictable (at low levels), and maintaining confidence at higher levels of abstraction through insecurity about everything falling apart". As in, "Of course they're closed to the idea that the CIA isn't messing with them. The alternative is to believe that their brain is failing to make sense of things. And if they let that in they might fail to make sense of anything -- because if they can't expect their brain to work then that opens up uncertainty faster than they can resolve it. And because if they can't expect their brain to work they can't intend for it to work, and if they can't even try to make things make sense then they're kinda SOL. They may have delusional beliefs, but at least they are coherent enough to be modeled as having beliefs".
Even if this isn't what schizophrenia is, this is the kind of thing you'd expect to happen if you had a brain that's just unable to make sense of its incoming data due to noise in the data -- perhaps including processing noise coming in at higher levels of processing. If your brain physically isn't working, you're going to have a hard time without a physical fix. But still, "The CIA is after me" is a belief, we disagree with it, and the difficulty of getting that to respond to evidence sure looks like a security limitation.
Recognizing that, and turning towards the uncertainty of how necessary that flinch is -- and how to make it less necessary -- provides a path to explore for anyone looking to help someone with this kind of problem find enough security to engage with the reality of their situation -- and therefore a more effective way of dealing with this likely shoddy hardware.
What would it be like for you if you were observing things that looked like the CIA were after you, and also like you just can't trust your own brain because you're losing your mind? "What to do if you're losing your mind?" is a terrifying question to ask, because you need your mind to answer it, and you can't trust that. Put yourself in those shoes, figure out how you could handle being in that situation, and you might have a path to helping the schizophrenic deal more effectively with their genuine hardware struggles.
Or maybe not. Again, this part is empirical, and on you. The point is just that once again, even when hardware is malfunctioning and making sense-making hard, there will still be disagreements to navigate -- just more of them, and with more difficulty of navigation.
Hardware doesn't have to be banged on or diseased in order to fail to perform. The same way a noncripple will kinda act like one after forced to run to the point of exhaustion, our brains can only do so much. We all have our limitations.
One of the more interesting experiences I've had in the last few years is of losing my mind. Partially.
I was dealing with a complex, high-stakes situation involving things very dear to me that required my full attention and processing capacity for over a month straight. I had to basically learn a new field, find results that the experts couldn't find, and then manage to earn the security/respect/attention needed to get them to accept my answers despite not having time to hash out our disagreements and explain how I knew my answers to be correct. This is on top of more than my normal life responsibilities. This went on for over a month, every day, without breaks. By the end, I started to break.
Normally, we maintain a fairly good agreement between what we believe and what we believe we should believe. We may have an irrational fear here or there, or be not quite as kind as we tell ourselves we should be, but more or less things will track. We make sure they track, by updating when divergences become apparent. I recognize concepts like "irrational fear" to be nonsense and "I should be nicer" as not necessarily correct (and how to find the extent to which it is justified), so I'm used to having very good agreement between my meta-beliefs and object level beliefs. As time went on though, I could feel myself coming apart at those seams; I just couldn't repair the gap as fast as it was growing.
My parents are great. Not perfect, because no one is, but great. They were very helpful at the time and certainly doing their best and engaging honestly. The only thing I could think to do when they got something wrong was yell at them, which was obviously crazy. I simply didn't have the ability to compute a more sensible answer on the fly, and had to physically leave so that I didn't do something stupid trying to engage with more than I could manage; I stand by and endorse that act of insecurity. Okay, whatever, people get stressed sometimes and yelling -- or better yet, walking away so as to not yell -- isn't that crazy, even if it's totally out of place. It's at least understandable, so long as it's a one off thing.
It wasn't just that though.
I left to the garage to work on a knife I had been making for my mom, gluing the handles on in the vice. And walked into the already sharpened knife. Normally I'm not so stupid as to walk into knives (I promise), but that day I didn't have the capacity for normal caution. Normally I wouldn't be so comfortable with what had just happened, but at the time all I could think was "Ooh! Human meat is red meat! Okay, where's the electrical tape". I had higher priorities. Places to be, things to do.
When driving to those places, I knew that I didn't have my wits about me enough to navigate there without error -- and it wasn't a complicated route, or a new one. I didn't even try to fix that, because whatever, driving safely is my priority and if I have to back track a bit after I miss a turn or two that's fine.
The scariest part, for me, was when I thought I was about to get in a fist fight, and couldn't do anything to stop it. Not scary because I was afraid of the fight, but because I was afraid of just how out of control I was of my own behavior.
Fist fights are fine as kids, but way not safe as full grown men, and absolutely not worth it. Normally I'm very good at deescalating by making it feel pathetic to lose control of oneself enough to take a swing in anger.[5] This time, when this guy started yelling in my direction, I knew very well that I would not be doing that. I didn't want to fight. Fighting is crazy, and I was still very very aware of that. At the same time, I just knew that if it turned out that it was me he was yelling at and he pushed the issue, I was going to make an equally persuasive case that he was pathetic if he doesn't swing. More specifically, I was completely unwilling to let him take my already scarce cognitive resources and redirect them to helping him shield himself from the destabilizing idea that being a middle aged man running around yelling at people for no reason is already pathetic -- even if he were to stop and get his shit together. I was just going to share my honest belief that he's beneath my consideration, and let him to do what he will with that information. It's just that I knew it was unlikely he could actually disagree, and unlikely that he was secure enough to update on this information, so the response he was most likely to have to that destabilizing information would be to try to make it go away by physically forcing me to recant.
It's hard to describe what the experience was like. I couldn't sort out that internal conflict in real time, and couldn't just self censor like I normally could. It was like I was a mere observer in my own body, with confident and legitimately justified meta-beliefs about what the right thing to do was, and it just didn't matter because that day I was not my elephant.
It's not that it "wasn't a disagreement", and it wasn't that it "couldn't be navigated" in some absolute sense. It took a month or so after it was all over, but I did return to baseline and would absolutely de-escalate that situation well today. So far as I can tell, every part of that is completely in line with the framework I offered here, and it's just a limit of how much information my brain could process.
At the time though, the result is that I just couldn't process more information. It's not that there wasn't time to say the words that point at the things, just that I couldn't process the emotional updates. You could have been freaked out and trying to warn me of the danger in the path I was going down, and I would have told you -- with an emotionally blank face -- that I agreed 100%. You're right. And it doesn't matter, because there's no place for that information to go and nothing for it to connect to. Take a number.
Take away sleep, nutrition, add processing noise, concussions, and everything gets harder. At some point I'd have to accept some losses in order to maintain sanity. The temptation was great, by the end of the month. When I finally did have to cut my residual losses it was the first time in a long time that I couldn't actually trust that I was making the object level best bet without being unduly swayed by temptation that I did not consider valid.
It's not that I couldn't update any beliefs, I just couldn't update them all.
I could have prepared for the unlikely event of some A-hole picking a fight with me, but it would have come at the cost of working on the more certain problems; I bet on it not being necessary. I could have let go of my anger towards arrogant idiots stepping out of line on whole, clearing a larger swath of potential problem, but that would have directly cost me the ability to get done what I needed done.
I could have learned something from walking into a knife, but again, it would have come at the cost of working on more important problems.
I did have a conversation with my parents, because it was important to keep that relationship working well in the short term too, but it took setting aside some time. I couldn't set aside enough time to cohere my beliefs enough to do it well, so I had to add a meta note of "I'm sorry I still sound kinda angry, I don't actually blame you I'm just legit struggling to hold it all together".
I had to prioritize. I had to track where I was flinching, which of my beliefs were wrong, and which disagreements were worth resolving when. This meant maintaining accurate meta-beliefs about my own inaccurate beliefs and their likely impacts, so that I could make reasoned choices with a brain that was physically incapable of updating on all the important information available to me -- and was gradually going crazy.
Thankfully it worked out.
I was able to mitigate the costs well and my bets paid off. I didn't get into any car crashes, or spurious fist fights, or damage my relationship with my parents. I do have a new scar, but it's small.
I got the results I needed to get, and I recovered my sanity. It was worth it.
Physiological limitations are real. Some of them influence the degree to which our brains can work as inference engines, satisfying the requirements for successfully navigating disagreements by "just showing relevant evidence". When we can't meet those requirements and update on evidence, then we just can't -- at least, without physically repairing our brains. Such is life, sometimes.
At the same time, even in those cases, knowing how to navigate matters. Sometimes it matters more.
While it has beliefs about how much power is "too much", it doesn't have any beliefs either way about whether this object level belief is correct -- and no place to put such beliefs and no ability to form them.
At a cost, remember. There's always a reason for things, and just because I chose to stop depressing there doesn't mean there wasn't anything to resolve, or that I was able to resolve it on the spot. Just that I decided on the spot that it wasn't worth trying to resolve at the time.
Applied to the nutritional-deficiency-induced-depression case, this would look like "let go, and be content to sit around with no energy -- until I can make it to a nutritionist" .
I know some people with similar experiences simply deciding to lose 70lb or quit smoking packs per day -- once they had a strong enough reason to do so.
Similarly, agentyduck explicitly works through the same thing for social anxiety. I'm particularly amused by this part:
"During the conversation in which I described my plan to him, we meandered to the topic of a meetup of professional hypnotists he’d recently attended. He told me they talked in passing about what it’s like to change their own behaviors. They all knew they could use a long, draw-out induction (or series of inductions and post-hypnotic suggestions) to self-modify if they wanted. But that takes time and energy, and it turns out that if you’re sufficiently confident it’ll work… you don’t have to bother with the hypnosis.
Think about that for a minute. They treated it as a perfectly normal, every-day occurrence. Basically they were saying, “Yeah, when I don’t like what System 1 is doing, I just tell it to do something else instead. No biggy.” They seem to have this available as a primitive action."
Because yes, that's exactly what it's like. And you don't even have to be a hypnotist to do it.
Implicitly communicating "You're better than that, I know you are. You know you are too."
Even if you wouldn't have been without the nudge, it's hard to counter "No I'm not!". It's also hard to counter "Losing my temper is totally a grown man thing to do!". Especially when you know the other person is still going to be looking at you like he sees right through you.