When I try to empathize with that woman, what I feel toward her is disgust. If I were in her shoes, I would immediately jump to getting rid of the damn nail, it wouldn’t even occur to me to not fix it.
You may not be doing enough of putting yourself into her shoes. Specifically, you seem to be putting yourself into her material circumstances, as if you switched minds (and got her memories et cetera), instead of, like... imagining yourself also having her world-model and set of crystallized-intelligence heuristics and cognitive-bandwidth limitations.[1] Putting your inner homunculus in the place of her inner homunculus.
One of the complications in agent foundations is that a bounded agent's effective action-space is much more limited than its physical action-space. There is, for example, a sequence of keystrokes you can execute right now that would result in your outputting the code of an aligned AGI. This is an action that is technically available to "you" – but it's obviously not available to you in any relevant sense. The failure to model that would be a failure.
It's subtler in cases where, like, "some part of the person knows what they need to do to fix their situation but they can't bring themselves to admit it and do it", or whatever. It's not as straightforward as directly missing knowledge; not isomorphic to lacking the key to an encrypted file. But I think it's directionally similar: they're not making decisions in the decision-theoretic landscape you model them as occupying.
From the perspective of someone with a nail stuck in their head, the world does not look like there's a nail stuck in their head which they could easily remove in order to improve their life in ways in which they want it to be improved. The world looks like a confused muddle, a constantly shifting dreamscape. They're not sure whether the nail is real, whether the nail can be removed, whether removing the nail would help, whether removing the nail is what they want. Their models of all of those things also keep wildly oscillating on a day-to-day basis: they can't act as a coherent agent over long horizons because neither their world-model nor their desires are stable. And even when they happen to be in the "removing the nail will fix it!" mental state, they're often just unable to make their body execute those motions: they lack the willpower/executive function/whatever-mental-resource to act on such plans, even when they manage to form them. Which, I think, is closer to "being unable to raise your arm because your muscles are crucially fatigued" rather than to "a personal choice/failure".
They're best modeled not as agents who are being willfully obstinate, but as people helplessly trapped in the cognitive equivalents of malfunctioning motorized exoskeletons.
Or, another frame, which is pretty uncharitable but is probably directionally correct: imagine them as yourself with brain damage. You can't move from the brain-damaged mind-state to a pre-injury mind-state by a sequence of normal human mental motions. Similarly, it's not a set of actions actually available to these people, for them to "snap out of it" and become John-like.
Don’t go bullshitting me about how a kind and compassionate life of mediocrity is a “different kind of strength”
On that note: Based on what I wrote above, one might expect me to be imbued with some sort of heartfelt emotions-level universal love. But... no, not really.
I do want everyone in the world to be happy and live good lives. Not only intellectually, I feel strongly about this. But those desires and feelings don't route through emotional empathy in me.
On the emotional level, I don't like most people. I find it easy to empathize with them in the deep way described above, but this does nothing to make me want to be friends with them. The "enjoying cats' company" mental motion doesn't come easily to me; my default emotions-level stance towards them is... disinterest.
Which is to say, doing empathy correctly indeed wouldn't necessarily make one kinder. It's possible to empathize with someone deeply enough to make all their faults understandable and their mistakes forgivable, and still not like them on a personal level.
But I do think you're failing to empathize deeply enough here.
Edit: Oh, didn't read the comments before posting, I see everyone already said that. I have more analogies though. :P
and if it's expensive it's because this is woven through a lot of other important decisionmaking processes for John
that is very interesting claim! why you believe it? my experience is that my aesthetics are part of ,y preference - not choose by me, almost impossible to change. I don't feel disgust, but i don't think i can switch easily if i decided so. in the same way i can't decide mechs are cool, or dragons are uncool.
There’s a narrative I hear a lot: if I empathize more, put myself in other peoples’ shoes, try to feel what they’re feeling, see things from their perspective, etc, then I’ll feel kinder toward them. I’ll feel more sympathetic, be gentler, more compassionate or generous.
And man, that sure is not my experience with empathy.
I usually relate to other people via something like suspension of disbelief. Like, they’re a human, same as me, they presumably have thoughts and feelings and the like, but I compartmentalize that fact. I think of them kind of like cute cats. Because if I stop compartmentalizing, if I start to put myself in their shoes and imagine what they’re facing… then I feel not just their ineptitude, but the apparent lack of desire to ever move beyond that ineptitude. What I feel toward them is usually not sympathy or generosity, but either disgust or disappointment (or both).
Like… remember the classic It’s Not About The Nail? You should watch the video, it’s two minutes and chef’s-kiss-level excellent. (Spoilers past here!) It opens with a woman talking about how she feels a relentless pressure in her head, she’s afraid it’s never going to stop, etc. Then the camera changes angle… and we see that there’s a nail stuck in her forehead. The guy next to her is like “Yeah… uh… y’know, there’s a nail in your head, I bet if we just got that sucker out of there…” to which the woman replies “It’s not about the nail! Stop trying to fix things when what I need is for you to just listen!”.
When I try to empathize with that woman, what I feel toward her is disgust. If I were in her shoes, I would immediately jump to getting rid of the damn nail, it wouldn’t even occur to me to not fix it. Sure, there have been times (though admittedly rare) when I want someone else to be sympathetic and supportive, but when I am not even trying to fix something myself I certainly do not expect sympathy from others. I can lean into the suspension of disbelief; if I were next to her, I could role-play as a supportive friend or partner. But put myself in her shoes? If I were in her shoes, behaving the way she behaves, I would feel disgust toward myself. She is not just inept in handling her problems, she lacks even the desire to fix that ineptitude, even when her problems are clearly imposing costs on both herself and those around her.
Another example: back in college, I had a long group project. And I liked my teammates, they were fun to hang out with while working on the project. Sure, I was the one doing most of the heavy lifting - not just the core technical parts, but also the writing, because both my relevant technical skills and my writing skills were in a whole different league from the rest of the team. But that was fine, I didn’t really think of them as people-who-were-supposed-to-be-helpful. As long as that suspension of disbelief was in place, no problem; it was an interesting project and I was happy to do it.
(I don’t think any of the people on that team are likely to see this, but if any of them do: this is the place to stop reading. Seriously, it will not do you net good to keep going.)
Then a conversation between myself and the professor overseeing the project dug a little too deep, and my disbelief temporarily ceased to be suspended. I had to look at the ineptitude of my teammates. What made it hurt wasn’t that they were stupid; this was a college where the median student got a perfect score on their math SATs, they were plenty smart. They just… hadn’t put in the effort. It was a machine learning project, and I was the only one on the team who’d studied any ML (years earlier; I knew well in advance that it would be a necessary skillset eventually, and already had experience with multiple other ML projects). Had I been in their shoes, I’d have at least gone through a set of ML lectures online before the semester started.
The disappointment came from seeing what they could have been, and seeing that they didn’t even try for it. They’d all come to one of the best colleges in the world, and then just followed the path of least resistance with minimal foresight for four years.
… then the conversation wrapped up, the suspension of disbelief went back into place, and I went back to enjoying their company.
I think a core factor here is something like ambition or growth mindset. When I have shortcomings, I view them as shortcomings to be fixed or at least mitigated, not as part of my identity or as a subject for sympathy. On the positive side, I have goals and am constantly growing to better achieve them. Tsuyoku naritai. I see people who lack that attitude, who don’t even really want to grow stronger, and when empathy causes the suspension of disbelief to drop… that’s when I feel disgust or disappointment in my so-called fellow humans. Because if I were in their shoes, I would feel disgust or disappointment in myself.
And then I put the suspension of disbelief back, and enjoy the cats’ company.
If you’re one of those people who wish people would empathize more, and believe this would lead to more kindness and compassion and gentleness and generosity… well, the main takeaway is for you. Consider that kindness and gentleness are not necessarily what everyone else feels, when they empathize.
And for those who share an experience more like mine… perhaps having pointed directly at the issue, you can now see a little better where others are coming from, when they ask for empathy. They don’t understand that empathy does not induce the things they imagine, for everyone. But empathy probably does induce kindness and gentleness and the like for them.
Lastly, for whatever smartass is about to suggest that my disgust/disappointment reaction is itself a problem to be fixed: only if it can happen in a way that makes me stronger, rather than weaker. I have no intention of lowering my standards for myself, unless that is somehow going to make me achieve more rather than less. Don’t go bullshitting me about how a kind and compassionate life of mediocrity is a “different kind of strength” or some such cope. But subject to that constraint, I would certainly like better ways of relating to people.