968

LESSWRONG
LW

967
Rationality
Frontpage

79

Follow-up to "My Empathy Is Rarely Kind"

by johnswentworth
31st Jul 2025
2 min read
40

79

Rationality
Frontpage

79

Follow-up to "My Empathy Is Rarely Kind"
39Ulisse Mini
18Ulisse Mini
12Ulisse Mini
2Garrett Baker
8johnswentworth
1Ulisse Mini
24David Lorell
12Thane Ruthenis
4James Camacho
2David Lorell
4James Camacho
14Kabir Kumar
5Viliam
14Jon Garcia
12fixwit
10CronoDAS
3johnswentworth
1fixwit
11the gears to ascension
8jimmy
6CatGoddess
5Ben Livengood
5Cole Wyeth
13Kabir Kumar
4Noosphere89
4Eli Tyre
4DirectedEvolution
3TristanTrim
4solhando
3Slimepriestess
3Jordan Rubin
3Pretentious Penguin
3Satya Benson
9johnswentworth
1Aprillion
1Myron Hedderson
1Myron Hedderson
5Mo Putera
1Myron Hedderson
0ekaterimburgo735
2[comment deleted]
New Comment
40 comments, sorted by
top scoring
Click to highlight new comments since: Today at 6:25 PM
[-]Ulisse Mini2mo3917

John, I think you are still missing something regarding empathy and it would be good for you to be open to that possibility. This post is a nice clarification but it still makes me think you don't get the thing in the same way I used to not get the thing with my ex. "Suspend viewing them as an agent" is the type of thing I also did, and yes, I could model her somewhat, but I was not really getting things emotionally.

I don't really view anyone as an agent anymore, some are more agenty than others, and wanting to mostly spend time with agenty people is fair, I don't think it's healthy to think about it this way.

Sure some people are cats compared to other people. Some neural nets happened to get better training data than others and have better initializations. Disgust and disbelief towards normal people is really not healthy imo, you shouldn't have to suppress or suspend anything.

Reply1
[-]Ulisse Mini2mo1811

It's hard to rationally convince someone of this, but you show some signs of missing stuff similar to not seeing colors re emotions. I'm not certain, but I think you'd would derive a ton of value from talking to a good coach/therapist re: empathy, emotions, possibly relationships. Idk if you know David Yu (co runs sparc) but he's who showed me the way I wasn't doing real empathy in a way that was intuitively grokked.

You're an exceptional alignment researcher but regarding relationships and emotional maturity I think you're highly underinvested & it's obvious to ppl who've invested more into those (such as me, obsessing over relationships and emotional stuff for the last ~6mo after a horrible breakup, a lot of coaching, meditation etc.)

Note: I'm not just indexing off the empathy posts, it's also "the value proposition of romantic relationships" post, this is something which most people intuitively feel relatively early on and don't need to derive. Several other signs too, such as not noticing you were depressed, again I'm not certain! But it's definitely worth exploring for you under uncertainty.

Reply1
[-]Ulisse Mini2mo1210

LW generally doesn't seem to value emotional intelligence and relational maturity very highly relative to intelligence and agency. I was similar, but creating a toxic situation which hurt the person I loved the most in the world totally changed my priorities regarding this. If you reading feel similar, "oh this isn't that important for me, I'm busy" consider unsong and that your robin could fall by your own deluded immature hands, without you even realizing it's happening.

Reply1
[-]Garrett Baker2mo20

consider unsong and that your robin could fall by your own deluded immature hands, without you even realizing it's happening

I can infer the meaning of this metaphor in context, but I don't remember any robins in unsong!

Reply
[-]johnswentworth2mo80

Mild spoiler!

The Comet King's wife's name was Robin.

Reply2
[-]Ulisse Mini1mo10

Comet King's wife

Reply
[-]David Lorell2mo2415

I'd personally agree that something like "suspension of judgement" is a key component to what I think is usually meant by "empathy."  I think for many people this does not go-together with a significant dampening of care or personal investment (in the short term at least) for/in the other person. It sounds like these do go together, necessarily, for you? Is that right? (Most (but not all) people usually care significantly less about cats than humans, especially family/friends/partners.)

Edit: On reflection I think I am doing the same / something similar in my orientation toward most people. (Suspending belief in moral agency / responsibility in the right sense.) Only for me, the mental move feels warm and fuzzy and kind as opposed to the misanthropic tone of the other post at least. I'd internally label it as closer to "forgiveness" and "charity" and maybe "faith/hope" in others.

An aside: When I imagine the nearest mental state I'd need to be in to systematically feel contempt instead of "forgiveness", it feels like I'm sliding into "should-world" and working myself up over how everyone (including myself in certain ways) is acting so stupidly and should just be doing X, Y, Z, with the telltale should-world feeling of ignoring the fiddly details and imagining things and people are meant to behave according to my values moreso than is probably reasonable.

Reply
[-]Thane Ruthenis2mo120

I think I have a thing very similar to John's here, and for me at least, it's mostly orthogonal to "how much you care about this person's well-being". Or, like, as relevant for that as whether that person has a likeable character trait.

The main impact is on the ability to coordinate with/trust/relax around that person. If they're well-modeled as an agent, you can, to wit, model them as a game-theoretic agent: as someone who is going to pay attention to the relevant parts of any given situation and continually make choices within it that are consistent with the pursuit of some goal. They may make mistakes, but those would be well-modeled as the foibles of being a bounded agent.

On the other hand, people who can't be modeled as agents (in a given context) can't be expected to behave in this way. They may make decisions based on irrelevant parts of the situations, act in inconsistent ways, and can't be trusted not to go careening in some random direction in response to random stimuli. Sort of like, ahem, an LLM.

Note that I think it isn't a binary "There Are Two Types of People" thing: the same person can act as an agent in some contexts and fail at this in others. That said, there is a spectrum of "in how many contexts does this person act as an agent?", and meaningful clustering around "not very many", "increasingly many", etc.

By itself, this is mostly unrelated to how much I care about a given person for the purposes of e. g. wanting their life to be better. (Like, I don't think non-agent-approximating people have less qualia or something.)

This is relevant for the purposes of who I would want to be friends/allies/colleagues with, for the straightforward reason of "people who are more well-modeled as coherent agents are more reliable allies", and also because it's a character trait I like.

Reply21
[-]James Camacho2mo43

People regularly judge themselves, so if you're really in their headspace, you would be judging them as they do (e.g. all their insecurities). Often, you'll come back to your headspace and think they're probably judging themselves too hard. But sometimes you don't. Sometimes you realize they're being way too lenient on themselves. As the car crash instigator often justifies, "it just came out of nowhere! It's not my fault, I couldn't have done anything better." Really? Or were you actively making choices that put you in this situation, and tried to avoid thinking about those choices every day until it became a habit and you no longer even had the capacity to realize where you were messing up?

I think the difference between John and 'normies' is that John is worse at deceiving himself. He holds himself accountable when things go wrong, and has trained himself to search for where he can improve, not just hope it magically happens someday. This is where the suspension of moral agency comes from: if others were truly morally capable agents, they would be trying to do the same.

Reply
[-]David Lorell2mo20

Very possibly I'm misunderstanding this but reading this comment felt like it missed the point of what I was trying to say. I find myself agreeing with most of what you say and not seeing why you've said it.

Reply
[-]James Camacho2mo40

Hmm, I felt like your comment pre-edit was saying, "empathy requires suspension of judgement," while I feel like what makes you feel disgust as you empathize with some people is because, "empathy requires judgement, and you're better at it than them." I think normies say, "empathy isn't judgement," because the act of empathizing helps them eliminate unfair judgements, and often even judge the other person kinder than they judge themselves. I think smarter people, who are brainwashed to think empathy should somehow end up at this conclusion, modify it to "empathy requires a suspension of judgement," but that's totally different from what most people are doing when they empathize.

Reply
[-]Kabir Kumar1mo140

what i remember of your post, made me think of cases when smart people are raised by people who don't know how to raise smart kids to know that others arent like them and how to get on despite that. 

it's like a tall guy going - why do people keep needing to get ladders for stuff, or keep not being able to dunk or reach things?? 

Reply
[-]Viliam1mo53

This, but when the tall guy says "oh, I finally get it -- everyone else is shorter than me!", everyone is shocked and insists that this is a horrible thing to say or even think, and the tall guy should be really ashamed.

(Or when he writes an article saying "I think it makes sense if I model other people like myself, but shorter".)

Reply21
[-]Jon Garcia2mo140

As far as I understand, "empathy" involves correctly modeling another person's feelings about a particular situation, and then taking on those feelings as one's own to some extent. This causes us to feel the same motivations as the other person and generally facilitates alignment of goals, assuming similar models of cause and effect.

When people say they want to be empathized with, I think it's usually mostly about the shared feelings aspect. Humans don't like feeling alone, and just having someone there who understands and accepts them, acknowledges their feelings without rejecting their legitimacy, can be comforting.

But there is definitely often a desire to have others take on their own world model as well, to take those shared feelings and direct them toward a shared goal. "I feel sad or angry about something, and I want this to be done about it!" An empathic person who nevertheless understands reality better and sees exactly why this hurting person's idea would never actually work can still acknowledge their feelings, but they may try to help the hurting person uncover what they actually need rather than what they think they want.

People's needs matter. People's feelings matter. But people's solutions and policy ideas often don't matter (or rather, they wouldn't actually solve the problems that cause the feelings).

Use your empathy to absorb people's needs and feelings, but use your rationality to direct the world toward actual solutions.

Reply
[-]fixwit2mo120

I think suspending moral agency is the wrong move. You say that not suspending it invites feelings of disappointment and disgust. I think those feelings are where you’ll gain the most by challenging. Why those feelings and not others?

I used to have issues doing chores; for example, I would regularly let my laundry pile up on the floor while my hamper sat next to the dryer full of all of the clean clothes I didn’t want to hang. It took me many years to overcome this, and for most of it I wasn’t trying to. At the time, it felt like quite the large nail in my head.

Was the solution a habit tracking app that gave reminders? Nope! I just ignored the reminders. Was it having others hold me accountable? Nope! I just got used to disappointing them with regard to it until they stopped trying to help. And so on, with many long stretches of not-trying in between. But one day, the feeling arose that “I should really remove this damn nail from my head,” and it was gone forever (or, so far). Importantly, that was maybe the thousandth time I had that exact feeling!

When I look back at myself at that time, I don’t feel disappointment or disgust, or the desire to view my past self like a cat (with regard to that context). Instead I feel curiosity—what were all the things going on over that time that I didn’t see? What emotional work was I doing that got me to finally remove the nail? Why couldn’t I remove it sooner? And those are some questions that bring me a feeling of kindness. 

Reply2
[-]CronoDAS1mo106

You needed one hamper for clean clothes and a second hamper for dirty clothes 😆

Reply
[-]johnswentworth1mo30

+1, that's what I do.

Reply
[-]fixwit1mo10

TRUE

Reply
[-]the gears to ascension2mo111

I do not suspend my thinking of cats as moral agents when considering their actions, and generally try to treat them as peers in that regard. I have found that treating cats as tools is an unreliable move. I do my best to communicate my intentions to cats, to express structured disapproval when they do something I don't like and try to explain why, etc. I once had an experience where a cat was biting me too hard, so I stood in front of him and bit myself too hard and then cringed; then bit myself very very gently and sighed happily. Then I offered to let him bite me a few times, and turned away from him to show my disapproval when he bit too hard, and slow blinked when he bit nicely. I gave him treats at the end. Reinforcement learning, but I tried to make it easily detectable to a cat brain why there was reinforcement learning happening, in the expectation that he could then use that information to decide what to do. I also never repeated this and for a long time he didn't try to love-bite me, even though he seems to really like lovebiting.

I think there's a way to look at these things which is something vaguely akin to importing both "empathy" and a form of the see-them-as-an-agent thing, something like "helping them to be a good person by both of our lights", which can take the thing that's currently rendering as disgust for you and direct it in a way that can be transmitted to them such that they'll be able to act on it; something like, encode it in a way that is better-lubricated by nature of your not treating them as a tool, and in fact having sympathy for the fact that facing the full brunt of what you want them to be might hurt at first, or some such thing. as well as including in the encoding of the request-to-change that it's not a thing where you claim to be the "true agent" here, that you're a peer not an authority. (well, unless you want to claim to be an authority, in which case, like, alright whatever).

In other words, or maybe just in very similar words again because I don't feel like I've said this clearly yet, I think there's a region between "treat as tool" and "treat as a peer who should already have chosen to do better", something like "present clearly that you'd like to treat them as a tool because you're disappointed in their effectiveness and request that they try to take on some of the effort of agency-towards-morally-good-outcomes that you're having to do through them".

If you were to do this to me, for example, just being disgusted, eg "I'm disappointed in your progress. You've been avoiding the hard work of doing the exercises that would teach you this math, and you're not at my level yet." that would likely make me averse to interacting with you - you're telling me things I already know and part of my brain yells at me about regularly, and the problem is not that I don't think that this is bad, but rather that I oscillate between going too hard at learning to having a feeling of "wow, doing the actual work is hard, can we budget the time spent on it?"; but if you were to say, "I'm disappointed in your progress, and to get best results from you as a contributor, I'd like to improve it as much as possible. What would it take to do that?" then you're asking me to narrate the bottlenecks I know about that are interfering with my ability to go hard.

Maybe I'm a bad test case, though - I already have a lot of motivation to go hard, and getting it to happen is where the problem is. If you're interested in saying this to people where your disapproval is pointed at the fact that they seem to reflectively endorse not caring to save the world even though they know the arguments for it, then I still think it would be a reasoning error to treat them as hopeless, but you'd need to do more things focused on achieving willingness-to-try.

Reply
[-]jimmy2mo82

People were talking about how critical the comments were of you, and I noticed that it hadn't stood out to me because the way you write it doesn't seem like it's hard for you. Or maybe it is, but you just don't let it get in the way. Either way, I find that cool, and admirable. I hope others see it too, and work towards the same thing.

Anyway, on the object level what I'd say is that people want to be seen as what they are. Removing suspensions and working towards full engagement does indeed bring teething issues, as you're experiencing, because not everything you learn with be toothless. Sometimes you do have to change behaviors in ways that hold people more accountable, and often that comes with an "oof".

And people really don't want to be almost seen for what they are without the full picture coming through (e.g. You don't want people to see the embarrassing things about you without also seeing why they'd have done the same thing in your situation).

At the end of the day though, full engagement with the kinks worked out does run more smoothly, and kindly, and is a goal worth aspiring to.

Reply3
[-]CatGoddess1mo60

Disclaimer: I want to start by saying that, since this is a hard subject to discuss precisely and we might have slightly different vocabulary, I might be going full steam down a side path and at some point in this comment miss my exit. Also, some of the things I say might be annoyingly obvious to you, but I thought they were worth mentioning in case they aren't. 

You said in the original post that you wanted to better relate to other people, which seems vague because you don't specify what you mean by "better" (by what criteria? what are you looking for?). However, assuming you want things from relationships that are similar to what other people usually want from relationships, my guess would be you wish you could more deeply connect with other people and maybe you feel a sense of alienation/loneliness. I can imagine someone feeling similarly unfulfilled if they only hung out with literal cats, even if they liked cats and had access to books.

Usually feeling isolated means feeling like you don't understand other people that well, and/or like they don't understand you very well, but in a specific sense. There are probably some ways in which you understand the average person better than they understand themselves or better than other "normal people" understand each other, but there's a difference between being able to predict their behavior or knowing what factors led them to be the way they are (e.g. having more knowledge of evolutionary biology) and relating to them emotionally.

To some extent that's a difficulty shared by many unusual people (e.g. veterans, expats, people with strange personalities), but I also think it's sometimes possible to emotionally relate to people more if you try to think a lot about them and make analogies between things you feel and things they feel. For instance, I'm not a drug addict, but I have on some occasions stayed up very late playing Civilization V while my inner homunculus tried and failed to get me to go to sleep instead, which is sort of similar in some ways to being a drug addict.

Another approach to relating to other people is to change things about yourself; if you became less ambitious you'd probably feel more emotional resonance with unambitious people. Likewise, if someone thinks of themselves as the victim of their circumstances whereas you take responsibility for rising above any bad things that happen to you, you could relate more to them if you started thinking of yourself as a victim of your circumstances. I think people sometimes do this, but I would guess you are not very open to that sort of thing; I would less confidently guess that you don't want to deliberately become more normal even in ways that don't affect your productivity in order to relate better to other people.

There's also the flip side where other people don't understand you very well, and probably often don't make that much of an effort to understand you better, which I suppose you have less control over. The second one especially seems hard to influence; maybe you could ask people to try harder to understand you, but I don't know that that's likely to work. It's somewhat possible to find common ground where other people can easily understand you (e.g. I like spaghetti and many other people also like spaghetti) and put in deliberate effort to "feel seen," which might work to varying degrees.

I think the upshot is something like, if you are feeling a sense of alienation/loneliness from lack of deep mutual "understanding" (in the sense I described it), there are probably ways to reduce that without compromising your values, but I'm also not sure how feasible it is to make it go away to a fully satisfactory extent. I believe the common wisdom is that it isn't, and that the only real solution is selecting better people to connect with.

Also, I think sometimes when people are talking about empathy they're referring to a sort of emotional porosity that doesn't route through conscious thought (e.g. feeling in higher spirits when you're in a room full of people in high spirits); you didn't mention anything like that, so I'm not sure much you experience it or know about it. I think sometimes this is a quality people like in others, and which enhances bonding. I don't know how much this can be deliberately controlled (to some extent it might be down to neurotype), but I do notice that when I'm at a rock concert I can either choose to get "swept up" in the collective excitement, or to hold myself apart. 

 

It also seems possible that your relationships with others are less good than they could be because of interpersonal mistakes you make - I wouldn't know, because you don't really go into details on things like how you treat people you view as moral agents vs people you don't; if it's basically the same as how you'd treat a cat I think people might reasonably take issue (also this is sort of a nitpick but "moral agent" seems like a strange word to use in this context because you talk about whether you see people as bearing responsibility for their own happiness, but it seems weird to call failing to be as happy as you could "immoral" - I'd intuitively just call that something like "agency").

Some examples: say Alice and Bob are friends. Bob believes he knows more about Alice's preferences than she does, which may or may not be true. He might sometimes insist that her stated preferences are wrong, and be unresponsive to her claims to the contrary. Alice probably finds this very frustrating, especially if Bob is in fact wrong. Part of the issue is that Bob isn't modeling her very accurately, and I suppose part of it is that Alice wants to be seen as an agent - but it's in the narrow sense of wanting him to believe that she's capable of knowing her own preferences.

It's also possible that, even if Bob has a relatively good idea of what Alice prefers, or a better idea than her of what's "good for her," she'd rather have the autonomy to make her own decisions - even if those decisions end up leading to less happiness for her; she'd rather he didn't override her autonomy whenever he has the opportunity. This is probably a situation where Alice would feel like Bob doesn't "see her as a human" - with the reason maybe once again being that Bob is failing to model her preferences. However, it also seems possible that maybe Bob is well aware that Alice places more value on autonomy than some notion like "welfare" but doesn't see that as important - in which case maybe it's more about what he values. This seems more like something to do with "respecting boundaries."

There could be this additional (perceived) violation where maybe Bob thinks she would benefit from going to therapy for some issue, but she's not very receptive to the idea. She might disagree that she actually has that issue, or maybe she isn't open to therapy for whatever reason. Bob is able to come up with schemes to get Alice to go to therapy without her realizing he's doing so (like, getting her mother concerned about her wellbeing, watching a movie with her that he claims he's interested in purely for its entertainment value but which promotes the idea of going to therapy, etc.). I think if she found out she'd be upset in this "I'm not being seen as a human" way, and it seems similar to this sort of manipulative element to people often treat cats and children (which seems probably fine for dealing with cats but maybe bad for dealing with children - at the very least I sometimes noticed my parents were trying to do this sort of thing to me growing up and I think they probably shouldn't have).

(Also, sometimes people manipulate children or cats not for the child or cat's benefit but for their own convenience, which seems worse.)

There's also an only somewhat related phenomenon where Bob positions himself as the "authority" in some sense and enforces that through a power differential; if Alice perceives this as unjust I think she might feel dehumanized in some sense. To give a clear example, when I was a kid my father would often criticize my taste in movies, video games, music, etc. in a very dismissive way. I wasn't really able to fight back or criticize his taste in turn, even though I think (even looking back on it now) his criticisms were in fact often based on a flawed understanding of the things I liked, and in cases where there were actually issues with my favorite media there were similarly egregious issues with his favorite media. 

This dynamic was enforced through a disparity in how well I could articulate my thoughts vs how well he could articulate his, and through the power he had over me as a parent who could punish me for things like "disrespectful tone"; I found it very upsetting. Going back to Alice and Bob, I think Bob can enforce a similar dynamic solely through superior intellect, ability to think quickly when stressed out, etc. (and sometimes this sort of thing happens in real life), though it would probably be more difficult.  

To be clear, I'm not accusing you of doing any of these things, but it seemed from what you've written in this post and the previous one that sometimes there's an element of interpersonal conflict or other people being upset with you, and the things I listed are broadly elements of how people frequently treat children and cats. It seemed worth mentioning in case you do behave in these ways sometimes, or in case other people perceive you to be. 

Reply1
[-]Ben Livengood2mo52

I'm curious what happens if you try on a different suspension of disbelief; imagine peoples' lives if they lack only growth mindset and not any other moral or agentic abilities.

I find quite a bit of difference in behavior between smart people who believe things about what and who they are, and the people who believe things about how they have acted, can change, and may act in the future.

Smart people without growth mindset often rabbithole into things like legalistic religions and overcoming what they perceive as unalterable weaknesses inherent to their nature, or try to maximize their perceived inherent strengths, ignoring development of new skills and abilities.  Introspection is like a checklist of how well they've done against a platonic ideal, with maybe some planning to avoid unwinnable situations.

Smart people with growth mindset usually focus on therapy (e.g. understanding their patterns of behavior and the outcomes and how they might alter those patterns in a persistent way), learning new skills and behaviors, possibly some mind-altering substances, and interacting with a lot of diverse other people to understand how they might change or acquire new beliefs and behaviors. Introspection is an exploration of possibilities and personal history and values and how to begin winning in previously unwinnable situations.

Less smart people tend to follow similar patterns, but slower or needing more guidance to proceed.

Reply
[-]Cole Wyeth2mo50

Where does this end?

I judge people pretty harshly, but that includes myself. In fact, in a way I judge myself most harshly, because I have to think about myself the most.

The vibe that I get is that John is usually talking about people who aren’t trying to radically self-improve. 

most of us are over that bar, and self-judgement is apparently part of the reason in my case at least. But it’s also, obviously, not enough. 

Reply
[-]Kabir Kumar1mo130

imo the place to end is accepting that different people have different values and priorities and different ways they want to live by them - and they're free to do so - the judging thing often comes from a false expectation kinda thing imo

Reply
[-]Noosphere891mo*40

My own take on this is that I do think a weak version of this is correct, in that empathy is not equal to kindness or value alignment, and I think a lot of comments on your last post are making a mistake similar to this.

Empathy's connotations need to be decoupled more.

That said, I'm going to slightly agree with the reactions, and a lot of this comes down to you often overestimating how much they are actually capable of changing the situation, and a milder issue is that you overestimate growth mindset.

The obvious example of this is IQ and human traits in general are pretty highly influenced by genetics, which we don't control, and to put it impolitely lots of success or failure is out of your control once we realize how important IQ is.

We can realistically only select, not train people, unless we get lucky with the motivations in their brain working out correctly.

People have some control over their lives, and that control matters, but we have a bad habit of assuming that we control more than we do.

Thane Ruthenis and Kabir Kumar's comments here are relevant:

https://www.lesswrong.com/posts/xPrL2xF9iYWpPmu6B/?commentId=7YscAPfjLYKdMP9qh

https://www.lesswrong.com/posts/KJh2xckmCNAggisuq/?commentId=YLE4ajQfpqTZMsqZm

Reply
[-]Eli Tyre1mo40

What traits can you observe in a person that causes you to regard them as a moral agent?

Reply
[-]DirectedEvolution1mo40

Do you really mean “suspension of disbelief” here? If empathy is suspending disbelief in another’s moral agency, that would mean that empathy involves temporarily indulging in a fantasy that they do have moral agency. That seems like the opposite of what you’re implying in the rest of the post.

Reply
[-]TristanTrim1mo30

Unless I badly misunderstood, what John is saying is that to be kind to people he needs to suspend his disbelief that they are not moral agents, as in, his default view is that people are capable of realizing they could be better, and trying to be. It is this belief in peoples "ability to be good" that he needs to suspend in order to be kind and friendly with them. If the suspension of this disbelief breaks and he starts to believe that it is within peoples power to be better than they are, then they are choosing not to be, which is pretty awful and strains his ability to get along with them.

I think he is then saying that empathizing with people is likely to break this suspense of disbelief leading to harsh judgment. This imo is something johnswentworth has stated as an observation of johnswentworth's mind. I don't know if he meant to imply that these judgements are always correct. I suspect not.

Reply1
[-]solhando2mo41

I agree with your first post.

However, I think the part of empathy that people will say you're missing is that you're putting "yourself" in someone else's shoes, which is only half of it. Imagine you were Isekai'd away into someone else's life where you had a history of being ineffective, unaccomplished, and self-pitying, with problems that could be solved quickly with relatively little objective effort. You can probably easily imagine how you'd quickly take actions to at least fix most of the low-hanging fruit in this situation. Clean your room, get a job, do at least some exercise, remove the nail and all that. 

But if you put yourself in the position of that person, including their internal mental state, possible brain chemistry issues, history of failure despite attempts to fix it, and possibly limited inherent capacities relative to yours, the situation they are in would probably seem a lot less fixable. If you put yourself exactly in their position, with the exact mental state and capacities, you would be doing the exact same things they are currently. 

For the woman whose pain is "not about the nail" there has to be something going on in her own head, whether it's some history of trauma, history of repeatedly failing to address the problem to the point it becomes painful to address it, that is stopping her from addressing the problem. Otherwise she would just fix it, no? To empathize with her isn't then to imagine yourself if you were that woman and had a nail in your head, but to imagine what it's like to be her, including whatever it is that's preventing her to solve her own problem. 

This sort of empathy might be more useful in understanding people, which can help you achieve your own goals better. There's always a need to make friends and influence people after all. Otherwise you're right in that putting yourself in other people's shoes (the ones who demand empathy are probably more likely to be pathetic), then seeing all the relatively easy things they could do to make their lives significantly better reasonably results in what you described with your first post. 

Reply
[-]Slimepriestess1mo30

Oh yeah, i do that all the time. if i don't im pretty likely to just start an argument with someone that ends with me crying. there's really not that many people i actually expect to possess moral agency by my standards, to the point where i usually interpret people who want me to hold them to my standards as trying to get me to help them self-harm.

Reply1
[-]Jordan Rubin2mo30

Empathy is a multi-dimensional operation and there are many ways to do it. Here are some examples of dimensions along which you can vary empathy:

  • How much do you try to literally “feel their feelings” vs feel your own feelings vs feel no feelings?
  • How charitably (or hostilely) do you try to read their situation?
  • How focused are you on actions they could take?
  • To what extent do you try to understand the story of what took them here vs focus on the present moment?
  • How focused are you on what is good for them vs on what is good for both?

    You don’t have to do empathy the same way in every situation. But have you ever tried to max out on “feel their feelings” or “charitable read”?

    Here is o3’s charitable read of nail-woman:
     
  • felt-sense override – pain is embodied confusion. before she can orient to “extract object,” she needs another nervous system to mirror how disorienting the signal is. otherwise her interoceptive map keeps screaming “unsafe,” so any fix attempt feels like an attack on the only reliable data she has left.
  • status & competence threat – accepting a drive-by solution telegraphs “i couldn’t notice a metal spike in my own skull.” that’s ego-annihilating. she’s fighting for narrative sovereignty, not against physics.
  • pattern-matching past dismissals – likely history of “just calm down” responses. the nail scene retriggers that cached pattern; she’s guarding against another instance of emotional rug-pull.
  • speech-act mismatch – she’s broadcasting affective data; partner keeps replying with instrumental commands. protocol violation feels like deafness. she insists “it’s not about the nail” bc rn the channel is meta-communication, not mechanics.
Reply
[-]Pretentious Penguin2mo30

Do you view moral agency as something binary, or do you think entities can exist on a continuous spectrum of how agentic they are? From this post and the preceding one, I’m not sure whether you have any category for “more agentic than a cat but less agentic than myself”.

Reply
[-]Satya Benson2mo30

From the previous post:

I think a core factor here is something like ambition or growth mindset. When I have shortcomings, I view them as shortcomings to be fixed or at least mitigated, not as part of my identity or as a subject for sympathy. On the positive side, I have goals and am constantly growing to better achieve them.

There is a tradeoff between this ambition and feeling at ease in the moment. Most people could probably use more ambition/agency, but I don't think it's clearly worse/worthy of disgust that many people don't care about growth enough to expend more than a certain amount effort towards it.

I'd be interested to know more about why you think you came to have relatively strong motivation towards achieving goals and whether you think that's ideal (even for people who value ease more than you do?).

Reply
[-]johnswentworth2mo91

I'd be interested to know more about why you think you came to have relatively strong motivation towards achieving goals and whether you think that's ideal even for people with different values than you.

I don't know how counterfactual it was, but the obvious branch-point was committing really hard to not let my values be overwritten by the memetic milieu (though that's not how I would have worded it at the time). Specifically, shortly before my twelfth birthday, I noticed that the people younger than me usually had grand dreams and ambitions (as children do), while the people older than me usually didn't. Evidently something was causing everyone to give up on their dreams, and whatever that thing was, it was rapidly approaching for me.

So I did the obvious thing and swore to myself as hard as I could that I wouldn't let whatever it was overwrite my dreams. And my dreams were not overwritten - though again, I don't know how counterfactual my precommitment was.

Reply22
[-]Aprillion1mo10

What were/are your dreams?

Reply
[-]Myron Hedderson1mo10

I think it's important to bring the distinction I initially missed, about what you mean by moral agency, into this conversation. From your comment in the other post:

I think this misses the distinction I'd consider relevant for moral agency.

I can put a marble on a ramp and it will roll down. But I have to set up the ramp and place the marble; it makes no sense for me to e.g. sign a contract with a marble and expect it to make itself roll down a ramp. The marble has no agency.

Likewise, I can stick a nonagentic human in a social environment where the default thing everyone does is take certain courses and graduate in four years, and the human will probably do that. I can condition a child with rewards and punishments to behave a certain way, and the child will probably do so. Like the marble, both of these are cases where the environment is set up in such a way that the desired outcome is the default outcome, without the candidate "agent" having to do any particular search or optimization to make the outcome happen.

What takes agency - moral agency - is making non-default things happen. (At least, that's my current best articulation.) Mathematically, I'd frame this in terms of couterfactuals: credit assignment mostly makes sense in the context of comparison to counterfactual outcomes. Moral agency (insofar as it makes sense at all in a physically-reductive universe) is all about thinking of a thing as being capable of counterfactual impact.

This seems defensible, but nonstandard. Under a definition of "moral agency" that relies on counterfactual credit assignment, a lot of things that would normally be considered the actions of a moral agent acting goodly, wouldn't count. (Unless I'm misunderstanding, in which case, please correct that misunderstanding.)

Examples:

1. I have an opportunity to cheat on my spouse, or do something else society clearly codes as wrong. I choose not to.
Standard interpretation: Good choice, have a cookie. You are the sort of person I can collaborate with.
Counterfactual credit analysis: Doesn't seem very agentic. Most of the credit here goes to the society around you and your parents, who taught you through various forms of reinforcement to decide in that way in that kind of situation. Maybe you get a little credit for actually doing the expected thing when the opportunity arose, but very little. You're basically a  cat. A good cat rather than a bad cat, I guess?

2. I want there to be fewer people dying of things they don't need to die of. So I read up GiveWell's stuff, and donate a large amount to each of their top recommended charities.
Standard interpretation: Again, good job, not many people do this at present and it's obviously helpful on the object level for people to behave in this way. 
Counterfactual credit analysis: ~No points awarded. Everyone with lots of money is aware of GiveWell now, their top charities are not funding constrained (at least that's what I understood to be the case a few years ago, don't rely on this statement as fact without double-checking), and if you didn't donate someone else would.

3. As a kid, my brother dies of cancer, so I vow to do what I can to make sure that happens to fewer people. (This didn't happen to me, but it did happen to a friend). I go study hard for decades and become a doctor specializing in the kind of cancer my brother died of (my friend did not do this, but he went a significant way down that path). Through various medical means, my actions directly prevent many deaths during my career.
Standard interpretation: Mission accomplished? You did what you set out to do, stayed true to the goals of your childhood self, and should look back on your life with happiness and pride.
Counterfactual credit analysis: The 80,000 hours career path analysis which basically said "the counterfactual impact of becoming a doctor is low, try and do something neglected" is where the concept of counterfactual thinking clicked for me. Few points awarded, clearly falls within the "just being a marble doing the expected thing" category of life-choices.

I think disgust at people who aren't agentic in terms of thinking about and optimizing for their counterfactual impact is the wrong move.

Background information that informs this view: Most people (not most people here, but most people generally) will have a reaction to "you're not really agentic and may get a disgust reaction from me unless you're optimizing for counterfactual impact" with either a blank confused stare because they're not familiar with the relevant concepts, or something like "you mean like speculating about how things would be different if the Nazis won WW2? What does that have to do with whether i should cheat on my wife, or get credit for not doing so?".

In this situation, most people are like the woman with the nail in her head, except she doesn't know she has a nail in her head and isn't going to be defensive about it unless you start telling her she's a terrible/stupid person or cat who you are better than and you can't work with her, for having a nail in her head and not doing anything about it. The standard person's reaction to counterfactual reasoning, once it's explained why that's relevant, might be "well that definitely changes some of my life-plans", which would be like the woman going "hey look, you're right, there is a nail in there, thanks!" Although consistently updating how you think to incorporate and apply a new concept in all areas where it's relevant rather than just applying it in its original context is also a skill that needs to be taught to many people, not something that happens automatically.

People who follow standard rules in standard situations are perfectly good collaborators in those situations. Also, reacting with disgust to those people regardless of their behaviour destroys the incentive structure which makes them good collaborators. People who don't cheat on their spouse and do become oncologists should get a cookie, even though those are standard things to do with a low counterfactual impact. And there is a material difference between someone who is a doctor and someone who is a nonfunctional alcoholic, in terms of how grown-up and reliable they are, and that difference should be recognized, rather than putting them both in the same bucket as a cat.

Another counterpoint: On my understanding of how physics works (which admittedly may be wrong in important ways) we don't have free will, there are no counterfactual universes, and we are all 0% actually-agentic. Your thoughts at moment t are a result of the various physical forces operating on the molecules of your brain, which strictly depend on the state of the system at t-1, and backward in time to the beginning of time. When pushed to the limit, "how much credit for this outcome goes to an agentic individual, vs. to environmental influences and marbles doing what marbles do?" has a correct answer "0% to the individual". Any choice of how much agency to grant someone in your mind seems kind of an arbitrary choice, unless you believe the universe is doing something other than unfolding according to physical laws, and our choices can in actual fact change the future. "Agency" is a useful component to a model of the interaction between humans and animals, not the territory of base reality. And picking "only agency that requires familiarity with concepts most people don't have familiarity with counts" seems like a choice that will have negative systematic effects.

With all that said... I agree with approximately all of this, from Thane, below:

The main impact is on the ability to coordinate with/trust/relax around that person. If they're well-modeled as an agent, you can, to wit, model them as a game-theoretic agent: as someone who is going to pay attention to the relevant parts of any given situation and continually make choices within it that are consistent with the pursuit of some goal. They may make mistakes, but those would be well-modeled as the foibles of being a bounded agent.

On the other hand, people who can't be modeled as agents (in a given context) can't be expected to behave in this way. They may make decisions based on irrelevant parts of the situations, act in inconsistent ways, and can't be trusted not to go careening in some random direction in response to random stimuli. Sort of like, ahem, an LLM.

Note that I think it isn't a binary "There Are Two Types of People" thing.

What you in your post mean when you say "grown up" in this sentence:

Sometimes they mean they want to be treated as moral agents (i.e. treated as a grown-up, rather than a child or a cat).

Seems to me similar to what Thane is pointing at with "game-theoretic agent", and what Harry in HPMOR would call a sane adult.

And, you get to choose who you will treat as an adult. If someone wants you to empathize with them, just do so, it's an almost-costless action that many people value. But if they want you to treat them as an adult, it's fine to say "sorry, there's a nail in your head and you should know it's there and want to remove it but apparently you don't, and under this circumstance I find it difficult to take you seriously". Not in those exact words, as that will quite often be perceived as an attack or extremely rude, but the message "there are requirements/standards if you want me to treat you as an adult, here is what they are, you're not currently meeting them" is useful information for people who want you to treat them as adults. Basically, the message "in order to be treated as a Very Serious Adult, you have to have your life and your self in a certain degree of order" is standard, even people who want to be treated with more respect than you're giving them may get it.

Reply
[-]Myron Hedderson1mo*10

Sorry about the long posts, but I'm thinking and trying to model how things look from your perspective. EDIT TO ADD: Epistemic status: Speculation.

Hypothesis: You're like, 30-40 points higher IQ than me, which would put you around 60 points above average (ballpark figures in each case). 

If true, that would explain some things. There's a certain intelligence level that I round down to "basically not intelligent", and a certain intelligence level that I round up to "too smart for me to really understand things they find intellectually engaging unless they try really hard to dumb things down, or I spend hours where they spent minutes, so I can't really have a conversation with them about it" - and in either case, it's hard for me to see distinctions among people too far from my own intelligence-level in either direction. And the same is true for everyone, from what I've read. The barrier to mutual understanding seems to kick in around 30-40 IQ points. I understand that for people at a certain low IQ level, "this person went to community college" = "this person is really smart", with the same reaction to "this person has a doctorate in physics" or "this person is the President". And I can talk to and connect with people who are around average, as well as people who are pretty smart, while I find it hard to really put myself in the shoes of someone who's significantly below average in intelligence, and there are people I tag with "too smart for me to really understand", although relatively few, and I can still understand the parts of them that, ahem, aren't particularly intelligent :D.

I picture what the world would look like if I was smarter, and thus concepts that took some prodding or prompting for me to get them (but I did get them) just seem obvious from age 5, the way utilitarianism did for me before I knew other people had thought of it and it had a name. Apparently this is something most people are only introduced to in university? Anyway, picturing what the world would look like if I moved up the intelligence scale, the thoughts that output sound like your posts. Most people are basically cats, if you expect to be treated like an adult you have to be trying to have a counterfactual impact. And my model of you as someone well outside the normal intelligence range predicts that my earlier post saying "counterfactual thinking isn't something most people get without being taught" would get a response like "yes, exactly, most people are basically cats, and I've just downgraded my estimate of your intelligence". The first part of which is a similar error to "a community college graduate and a top-level physicist are basically the same".

The more carefully-worded version of "counterfactual thinking isn't something most people get without being taught", would be something like "counterfactual thinking isn't something most people do without being taught, except in rare and fairly stereotypical circumstances, like 'I was just almost in a car accident' or 'what would my life be like today if I had stayed with my first love?'". I mean, yes, they do basic counterfactuals like "if I eat the cake I will get fat, if I don't eat the cake I won't get fat" (which, I note, cats do not), but thinking about the higher-order effects like "if I buy the cake that has this effect on the overall economy, and the world as a whole looks different 6 months from now in these subtle ways" is a thought-pattern most people have to be taught - but can be taught.

If your situation is that you can't differentiate between average-intelligence people, below-average-intelligence people, and cats, because they all just don't get things that seem obvious to you, and once they don't get one obvious thing you worry about what other obvious things they will or won't get and they just become unpredictable beings you don't understand very well... then probably my encouragement to treat more people less like cats isn't going to work for you.

Reply
[-]Mo Putera1mo50

Anyway, picturing what the world would look like if I moved up the intelligence scale, the thoughts that output sound like your posts. Most people are basically cats, if you expect to be treated like an adult you have to be trying to have a counterfactual impact.

This tangentially reminded me of this quote about John von Neumann by Edward Teller, himself a bright chap (father of the hydrogen bomb and all that):

von Neumann would carry on a conversation with my 3-year-old son, and the two of them would talk as equals, and I sometimes wondered if he used the same principle when he talked to the rest of us.

That said in John Wentworth's case moral agency/ambition/tsuyoku naritai seems more key than intelligence, cf. what he said earlier:

What made it hurt wasn’t that they were stupid; this was a college where the median student got a perfect score on their math SATs, they were plenty smart. They just… hadn’t put in the effort. ... The disappointment came from seeing what they could have been, and seeing that they didn’t even try for it. ...

I think a core factor here is something like ambition or growth mindset. When I have shortcomings, I view them as shortcomings to be fixed or at least mitigated, not as part of my identity or as a subject for sympathy. On the positive side, I have goals and am constantly growing to better achieve them. Tsuyoku naritai. I see people who lack that attitude, who don’t even really want to grow stronger, and when empathy causes the suspension of disbelief to drop… that’s when I feel disgust or disappointment in my so-called fellow humans. Because if I were in their shoes, I would feel disgust or disappointment in myself.

So I think you're misdiagnosing.

Reply
[-]Myron Hedderson1mo10

You could be right, and thanks for the feedback. It's a low-probability speculation, and that quote is evidence against.

There's a difference between disappointment and disgust, and "can only have fun with people when he treats them as non-agents" is very different from how I think about people, and it is in my nature to try and figure out people who think very differently from me. So far I haven't got a mental model that fits John's outputs well in their entirety. My mind is still working on it in the background.

Reply
[-]ekaterimburgo7352mo*00

Thank you for the follow-up post. The distinction between empathy and the attribution of "moral agency" is a very helpful update and greatly clarifies the crux of your original argument.

That said, your conflict doesn't seem to stem from simply attributing moral agency, but from an implicit assumption about the *utility function* that all moral agents *should* be optimizing.

If we model humans as self-optimizing agents, we can distinguish between:

1.  Terminal Goals: The final, intrinsic objective. I would argue that for most humans, this approximates some form of "well-being" or "satisfaction" (happiness, to put it simply). It is the utility the system is trying to maximize.
2.  Instrumental Goals: The subgoals an agent pursues because it believes they will help it achieve its terminal goal. This is where everything else comes in: strength (`Tsuyoku naritai`), competence, knowledge, wealth, social connection, validation, security, etc.

Your original post and this follow-up suggest that you have elevated a very specific instrumental goal—competence and personal growth—to the status of a universal terminal goal. The "disgust" or "disappointment" you feel when activating the "moral agency module" seems to be a reaction to agents who are not optimizing for *your* chosen instrumental goal.

The conclusion isn't that you should "lower your standards" for yourself. The conclusion could be that treating someone as a "moral agent" doesn't just mean demanding that they be responsible, but also recognizing that their responsibility is to their own, unique utility function, not to yours.

Reply
[+][comment deleted]1mo20
Moderation Log
More from johnswentworth
View more
Curated and popular this week
40Comments
Deleted by DirectedEvolution, 08/01/2025

There’s now over 100 comments on “My Empathy Is Rarely Kind”, and the consensus response is roughly “John, that’s not what empathy is”.

So, first things first: I agree. I was using the wrong words for the things I tried to describe, and this was a useful thing to realize, so thank you everyone for that!

As a result of using the wrong words, I think people mostly walked away with the impression that I don’t know how to do the-thing-usually-called-”empathy”. And while I don’t claim to be especially good at empathy, I think the post gave an inaccurate impression, due to using the wrong words.

I do, in fact, empathize with literal cats. I track what the cat is thinking and feeling, and part of that is vicariously feeling some of what the cat feels. I can empathize with humans in the same way I empathize with cats. That form of empathy involves keeping the “suspension of disbelief” in place.

But then… suspension of disbelief in what? If I am in fact empathizing (as the word is typically used), what disbelief have I suspended?

On reflection, I think I suspend disbelief in the subject’s moral agency. (This was a useful thing for me to realize, so again, thank you everyone!)

With cats, this is usually pretty easy. I don’t usually think of cats as the type of creature which it makes sense to hold responsible for anything. If working on a team with a cat, I wouldn’t treat it as another agent, I’d treat it as a tool or assistant. I wouldn’t particularly expect a cat to uphold its side of a contract, except insofar as that's a default outcome. I wouldn’t give a cat voting power. I do want cats to be happy, all else equal, but I don’t think of the cat as having any responsibility for its happiness. Or for anything else.

And that’s the sort of attitude I take toward other humans, when I suspend disbelief in their moral agency (which I do most of the time).

With that in mind, let’s go back to the opening of “My Empathy Is Rarely Kind”:

There’s a narrative I hear a lot: if I empathize more, put myself in other peoples’ shoes, try to feel what they’re feeling, see things from their perspective, etc, then I’ll feel kinder toward them. I’ll feel more sympathetic, be gentler, more compassionate or generous.

And man, that sure is not my experience with empathy.

People say they want to be “seen as a human”. And sometimes they mean that they want to be empathized with. Sometimes they mean they want to be treated as moral agents (i.e. treated as a grown-up, rather than a child or a cat). And I think I wrote an essay responding to the empathy one, when I should have written an essay responding to the moral agent one.

… but also, I think people jumble those things together a lot? Perhaps a more accurate takeaway from “My Empathy Is Rarely Kind” would be: I can empathize with people while suspending judgement, empathize while suspending disbelief in their moral agency, but that does not usually make me more sympathetic toward them when I do view them as a moral agent.

Mentioned in
90Generalized Coming Out Of The Closet
35A Self-Dialogue on The Value Proposition of Romantic Relationships