On sincerity

[-]paulfchristiano3y*171

Here is an account that tries to unify these associations. I mostly took this away from the post rather than coming in with it, and so I think it’s quite similar to what you are saying but maybe a bit cleaner.

Conversation or deliberation often has the explicit purpose of discovering truth or making a decision. But we also have a bunch of other reasons we care about how conversations or deliberations go. For example:

They take time and attention that we could spend elsewhere.
They can be fun and interesting, or boring and uncomfortable.
If public, listeners will infer something about us from what we say.
Even if private, thinking clearly about a topic can make it harder to maintain a fiction.
They can effect the balance of power in a group or within an individual.
Sometimes we just don’t want to help other people find the truth.

I’m inclined to call a deliberation or conversation sincere to the extent that we are prioritizing mostly the decision-making or truth-seeking function above all that other stuff we care about.

Any of those other considerations can lead to a failure of sincerity: I don’t want to spend the time or attention, I want to take the opportunity say something witty, I don’t want to show my true colors, I want to tell an explicit lie, I want to avoid challenging a convenient self-image, I want to avoid lines of inquiry that might make me feel compelled to do something...

I think that this mostly captures the associations you describe:

“Like truth-seeking” is straightforward.
“Not playing pretend” is straightforward.
“Healthy stapleclippers” becomes a particular failure of sincerity, where deliberation and inquiry are casualties of conflict between conflicting impulses—my sense of sincerity doesn't require that conflicting impulses get along in a deep sense of finding the best bargain or anything like that, but it does require that they be willing to come to the table and talk it out (or at least not sabotage when talking-it-out is occurring).
“Seriousness” also seems straightforward (and quite close to a synonym). There is a separate kind of seriousness that is more like “appreciate the stakes of the interaction," e.g. "watch what you say, offending someone could start a war.” But that's clearly not what we mean, and I think sincerity is close to a synonym for the other sense of seriousness (+ honesty).

This captures the sense of someone being insincere because they are unwilling to pay attention to the importance of what's happening. It doesn't really exclude as insincere the person who seriously considers the stakes and concludes that they care more about saving office supplies than saving people, but as you say I think this might just be a desire to lump all good things together and I don't think that there is any real sense in which the paperclipper is necessarily insincere. This leads us to...
Sincerity will tend to be correlated with “having something to protect.” If you just care about how fun your life is, it’s less likely that it’s worth taking the attention and time to engage in sincere inquiry. If you just care about status, it’s unlikely to be worth ceding ground in a conversation even if it would help you make a better decision. And of course if your preferences have no overlap with your peers' you are less likely to be honest.

But there’s no fundamental tension to being completely self-interested and completely sincere. And it seems easy and intuitive to imagine sincerity in the service of forward-looking self-interest, e.g. to imagine having a sincere discussion with someone about what kind of life they really want and what path is most likely to get them there. And in fact I think that's probably one of the most common kinds of sincerity.

Broader or more robust associations with altruism aren't really included on my account. That said, if sincere inquiry is inquiry that prioritizes figuring out what you should do, and if we imagine hypothetical reasons that would be moving to such inquiry, then I guess sincerity is whatever causes you to be moved by realist-style moral truth (though I’m skeptical of that hypothetical for some of the same kinds of reasons you discuss in against the normative realist wager).

Some other scattered thoughts:

I intuitively care a lot about people having space for sincere conversation and inquiry. It doesn’t seem like something you’d necessarily want to permeate your life, or that is necessarily a particularly satisfying way to be in the moment. It seems reasonable for most moments to not-even-try-to-be-sincere (as contrasted with insincerity which is more like the pretense fo security), as long as there is enough space for sincerity. You touch on this early in the piece but I think that I have a different vibe around it than you do.

Relatedly, I may imagine implementing sincerity in a different way from you, and I'm a bit skeptical about the ability of a human to be sincere by making a move like “just let go of pretense.” It seems like motivated cognition runs deeply in humans, both in practice and in theory, and we shouldn’t expect to have any kind of direct access to whether we are being sincere at any given moment. (Perhaps you can learn to notice, but it seems like cultivating sincerity based on this perception just seems likely to lead to more effective delusion.) For example, in the context of “should I be vegan?” I would be inclined to proceed lightly and with curiosity while exploring what is true, the potential arguments, the philosophical intuitions, what it would actually look like to be vegan in practice, and so on. And then only once all that is in place I might decide to really try to approach the decision itself with an open mind in a more serious and heavy way. But that’s exhausting and has high stakes and involves more material tradeoffs, and it’s not the space you want to be in while figuring out the facts. (I think this makes the inquiry sincere, but with a different mood and vibe.)

I care about having space for sincerity largely because it facilitates a certain flavor of important engagement. If someone makes space to engage sincerely then we can negotiate and trade, we can try to persuade each other, or we can collaboratively seek truth. Even if I imagine a sadist whose preferences are nearly the opposite of mine, if they are willing to have a sincere conversation then we can at least search for common ground, we can appreciate together the tragedy of our situation, we can identify topics where truth would help us both and seek it together. On the flip side there can be people whose values are almost the same as mine but who would not engage sincerely with argument or negotiation. And for that purpose I just wonder whether someone will have a conversation actually oriented around its nominal purpose, or whether (often legitimate!) considerations would make that impossible.

I think the cost-benefit analysis for sincere inquiry is a little bit more loaded and unclear than this post makes it out to be. (This was also a major objection I had to The Scout Mindset.) I think that even thoughts in the privacy of your own head have significant impacts on how you relate to the world and what you do, and it’s a substantive claim that either those costs are manageable or that the benefits of sincere inquiry are large enough to offset them. My own inclination is that it’s worth it to have significant space for sincere inquiry and at least some (and perhaps many) people with whom you regularly have sincere discussion, but I don’t think it’s completely obvious and I would probably approach it with a bit more trepidation than you would.

[-]Joe Carlsmith3y30

Thanks for these thoughtful comments, Paul.

I think the account you offer here is a plausible tack re: unification — I’ve added a link to it in the “empirical approaches” section.
“Facilitates a certain flavor of important engagement in the vicinity of persuasion, negotiation and trade” is a helpful handle, and another strong sincerity association for me (cf "a space that feels ready to collaborate, negotiate, figure stuff out, make stuff happen").
I agree that it’s not necessarily desirable for sincerity (especially in your account’s sense) to permeate your whole life (though on my intuitive notion, it’s possible for some underlying sincerity to co-exist with things like play, joking around, etc), and that you can’t necessarily get to sincerity by some simple move like “just letting go of pretense.”
This stuff about encouraging more effective delusion by probing for sincerity via introspection is interesting, as are these questions about whether I’m underestimating the costs of sincerity. In this latter respect, maybe worth distinguishing the stakes of “thoughts in the privacy of your own head" (e.g., questions about the value of self-deception, non-attention to certain things, etc) from more mundane costs re: e.g., sincerity takes effort, it’s not always the most fun thing, and so on. Sounds like you've got the former especially in mind, and they seem like the most salient source of possible disagreement. I agree it's a substantive question how the trade-offs here shake out, and at some point would be curious to hear more about your take.

[-]jsd3y10

Thanks for this comment, I found it useful.

What did you want to write at the end of the penultimate paragraph?

[-]paulfchristiano3y20

Thanks for pointing that out (I cut the hanging sentence)

[-]Harrison G3y30

(read: The Athena-Parfit Long-Term Institute for Raising for Effectively Prioritizing Global Alignment Challenges)

I laughed about this for a while. Thank you for this though-provoking post, and for incorporating occasional humor throughout.

[-]Slider3y20

This direction is a very common occurrence with spiritual oriented stuff that I encounter.

Consider to use the phrasings of "What shall I do?"(but not in the what I would like to do sense) and "What will I do?" (but not in the prediction sense). It seemed plausible to me that the deliverable is volition.

I will try to recreate some phrasing that I feel tried to reach at this same thing in the hopes the different angle helps.

Division leads to fragmentary action which will inevitably produce conflict. So as long there is the self and environment total action is impossible. (combined with shard theory, consider what happens if there are no net winner or losers within the shards or if the attribution system flags all the shards or if all shards bid in the same direction)

Think about being a whale saving campaign runner in your free time and being a whale hunter as your job.

Also a sketch about taking peace seriously which highlights some failure modes rather nicely.

-What papers have you got there?
-You see I am extremely concerned about the state of world peac. Now I have established a peace association named Peace Securers.
-Do you have to establish a new association? Aren't there lots of existing ones to join?
-But I don't accept their peace, I have to have my own. You see, the others are wishy-washy paraders walking with banners and singing protest songs. But my Peace Securers will establish peace to the world quickly and conclusively. If nothing else, with fist and dynamite, you get me?
-No
-Well sign up this joining form and you will soon understand
-No, I am not going to join any associations
[threatening collar-grip ensues]
-Don't you see, you don't play around with my Peace Securers. Name to the paper or you are going to be the first victim of the new world peace.

Making an identity means that you will be some specific sort of way. Which means some mode of action or being is now forbidden to you. If the situation calls to do that forbidden thing, you must choose between (that version of you dying + winning) and (surviving+messing the situation up). In order to not have such blockages you need to be nothing in particular.

Doing things myopically leads to doing two things that are contradictory. This is just a wastage of energy. "Weakness of will" scenarios involve you defeating yourself. The aggressor is the aggressed (aspects of it feels like "general society" but it can only infact be you who is doing the thing). If you do a choice reluctantly this will diminish or bother some part of you. Delving in enough depth to a question means not having such shadows-of-doubt. Digging to the truth-or-falsity of an insult will totally destroy its damagingness.

A conversational pattern with a question and guidance "do not take this as a simple claim that you either affirm or deny and then move on. Understand the question and think about it". The trick has a success branch where no answer to the question is produced (ie thinking about it, "getting serious about it" is success enough. Giving a cached answer is a failure. (note that with important enough questions people will avoid answering "I do not know")

A technique to get to know something. Explore the significance that you do not know that thing. The more serious the question gets, you start to apply a wider range of tools and keep at the search for longer.

You can not escape your conditioning. You can't magically choose to be perfectly rational or enlightened. You must find a step that you can actually, in your limitation, take (noble regret). Your conditioning has entered from the environment and has in large part has already got honed before you were around.

Instead of analysis, the very seeing of the way to act is the doing. The proof is a recipe in the same go. Realization has the "manifestation" side as well as the "understanding" side.

^{^}

For simplicity, let’s assume you have some good reason for this – i.e., you’re trying to help people, sabotage the Nazi regime, etc.

^{^}

Another issue is that “sincerity” can also sound a bit nicey-nicey. A bit saccharine. Again, a relevant connotation; but not ideal.

^{^}

Of course, in the real world, genuinely opening a question can itself require an investment of time and energy that someone may not be up for at a given moment, even if they’re up for it in principle. They may have “made a previous decision about that,” and prefer to move on rather than reconsider. This is indeed one way that conversation about a question like this can fail – but I’m hoping to set it aside.

^{^}

See here for one argument – though you also need the premise that being less-than-maximally effective (even just in helping animals) is wrong.

^{^}

Thanks to Katja Grace for suggesting, in another context, that I consider what answers a social circumstance “allows you” to give.

^{^}

See Yudkowsky, here, for related concerns.

^{^}

Thanks to Bill Zito for discussion of the “not asking” option.

^{^}

Alternatively: scout-mindset arises very naturally when you’re sincere about something.

^{^}

“Beliefs are for true things.”

^{^}

I owe the idea of the decision-making part being “in the room” to Anna Salamon, years ago.

^{^}

See my essay on moral demandingness for more on this.

^{^}

Here I’m reminded of this debate between Shelly Kagan and William Lane Craig. “Why do we stray so much?” the moderator asks, and Craig speaks about sin. And then – and here’s what struck me – Kagan talks about sin as well – a word that felt strange in the mouth of analytic philosopher. Maybe one objects to its theistic implications. But maybe it’s a good word, anyway.

^{^}

“No one who either knows or believes that there is another possible course of action, better than the one he is following, will ever continue on his present course” (Protagoras 358b–c).

^{^}

Compare with e.g. an alcoholic who claims to really want to quit. Would they take the “quit drinking” pill? That said, especially outside the context of straightforward addiction, intentional motivational self-modification like this is complicated, and it’s reasonable to be wary of it. But set up the case so that it’s maximally good on these fronts. For example, let’s say that it’s a temporary modification that will definitely be reversed, at which point the person will have a waiting period in which they decide whether to do it again; and let’s say their decision-making, in the interim, will be confined to low-stakes contexts with relatively reversible effects.

^{^}

See Bennett (1974) for a collection of quotes.

^{^}

Indeed, I think it’s important fact about evil that it is often done both knowingly and cheerfully.

^{^}

Thanks to Cate Hall for discussion, here.

^{^}

Indeed, many everyday cases of insincerity mix self and other-directed deception together. Someone is trying to tell some story both to themselves and to the world, at once: “I am a person of type X,” “the reason I’m doing this is Y.” Perhaps, indeed, the world’s opinion matters centrally insofar as it makes the internal story possible. See also Hanson on “Smart Sincere Syndrome,” his work with Simler on “The Elephant in the Brain,” and much else.

^{^}

Katja Grace, in discussion, suggested to me that if you’re fully wrong about your motivations, then in some sense you might recover a form of sincerity. After all, you’re not lying to yourself (or at least, not anymore; perhaps you tried, and succeeded, in the past); rather, you’re just entirely mistaken.

^{^}

Indeed, social roles are frequent sites of playing pretend, because they come both with a telos – an expected goal, a thing they’re supposed to be about – and with various forms of status, power, identity, and belonging that easily trigger other motivations. Science, we think, aims at truth; art, at [something complicated – beauty?]; public service, at the public good. And so, too, does the true scientist, artist, public servant. But how much of science is true science? How much of a “literary scene” has its eye on the telos of literature? But if one hopes to call oneself a scientist, or an artist, or whatever, there is an incentive to tell stories – including to oneself – on which one’s eye is on the prize.

^{^}

Though, also Lewis: “If you do a man a good turn, not to please God and obey the law of charity, but to show him what a fine forgiving chap you are, and to put him in your debt, and then sit down to wait for his ‘gratitude’, you will probably be disappointed.” Both quotes from Mere Christianity.

^{^}

See also Hamlet on “denoting me truly.”

^{^}

Here I’m actually thinking of a tradition subtly different from what I see as the first-cut model of Internal Family Systems, on which there is such a thing as The Self – a centered, open-hearted, peaceful version of yourself, which is implicitly treated as the “true version,” obscured by the distortions of your parts. On the picture I have in mind here, there is no “true self” that the parts distort; rather, there are just the group dynamics of the different parts interacting. More on this difference below. In practice, though, many of the mental moves are similar.

^{^}

No: “What I am trying to say is that more than your own life has to be at stake, before a person becomes desperate enough to resort to math.”

^{^}

We could try to construct one via appeal to the types of sacrifices someone is willing to make, where sacrifices are understood in terms of goods like your life, your time, and so forth. I discuss this below.

^{^}

I haven’t read much of Kundera’s The Unbearable Lightness of Being, but I suspect that sincerity falls on the “heaviness” side of his lightness/heaviness distinction:

“The heavier the burden, the closer our lives come to the earth, the more real and truthful they become. Conversely, the absolute absence of burden causes man to be lighter than air, to soar into heights, take leave of the earth and his earthly being, and become only half real, his movements as free as they are insignificant. What then shall we choose? Weight or lightness?”

^{^}

I’ve also heard “no status” – e.g., here.

^{^}

Thanks to Claire Zabel for suggesting an example in this vein.

^{^}

Which is not to say, of our own world (sinking or no), that one should forget all about pride and status. Indeed, I think it’s worth wariness of people saying “forget all about that status bullshit and do what I say.”

^{^}

This isn’t limited to consequentialism, or to demanding religions: it applies to common-sense morality as well. Thus, for example, we tend to think that if you’re driving to the hospital to save yourself from imminent death, but getting there in time requires running someone over on the way, morality prohibits you from running them over: you have to let yourself die instead. More here.

^{^}

Cf. Freddie DeBoer:

"…the only passion that matters is a passion for the ordinary, because we live in the ordinary. Nothing gold can stay. Provocations don’t stay provoking. You start out full of fire and if you’re successful in your professionalization goals you get an office and settle in to filing grant applications. That’s life. This is what the radical left can’t understand, is bent on not understanding: if you’re only into it when it’s exciting, if you can only get the energy to participate when you’re rioting in the streets, you’re not really into it at all. You’re just a tourist. Real left organizing is renting the Portapotties for a demo. That’s the work…

Here is a little bit of hard-won wisdom from someone who has often thought that he has found The Thing, the one overarching project or mindset or philosophy that will dictate the path of the good and the decent: The Thing doesn’t exist. You never find it. All Things eventually go the way of your Thing from when you were 20, when you discovered Buddhism/existentialism/Ayn Rand – it inspired and animated you, you thought you had broken the world’s code, then over time, the cracks inevitably showed, and what seemed energizing and limitless within that Thing was subsumed into the plodding ordinariness of everyday life. I am still a Marxist, as I think the Marxist theory of history (which leads inevitably to the Marxist theory of economics) better describes the world than any other philosophy I’ve encountered. But my adult life has been one long repetitive encounter with the limits of Marxism, not because it’s the wrong Thing but simply because it’s a Thing, and in fact my political origin story really starts at the day I walked away from the orthodox Marxist reading groups of my young adulthood."

^{^}

Lewis, in Perelandra: “he felt like a man brought out under naked heaven, on the edge of a precipice, into the teeth of a wind that came howling from the Pole.”

^{^}

Glancing at this link, I first thought this quote was from an Islamic source, but I think it’s actually from Saint Hildegard of Bingen.

^{^}

Perhaps the Christians (though not the rationalists) would say: some form of death. Bonhoeffer: “When Christ calls a man, he bids him come and die.” St. Francis: “For it is in giving that we receive, it is in pardoning that we are pardoned, and it is in dying that we are born to eternal life.”

^{^}

Though maybe some of the greeks thought this?

^{^}

“Joe, won’t you ever get tired of meta-ethics? Just let it go, man.”

^{^}

See e.g. here for an example attempt – one that I think fails.

^{^}

Video here, though: explicit, spoiler-ish, and potentially disturbing.

^{^}

Maybe you’re like: Joe, this is just a classic “True Self” vs. “No Self” debate, which is resolved via something something non-duality etc. But I don’t think so. In particular, neither side of that debate countenances the possibility that the spiritual ideal leaves you a sadist, egoist, paperclipper, etc – though perhaps, per the recent “enlightenment is compatible with being an asshole” vibes in some western Buddhist circles, both should.

^{^}

Of course, there are empirical questions here about what will happen to actual humans when they do blah form of therapy, about where actual patterns of sadism and egoism etc come from, and so on. But I’m holding off on empirics until the next sub-section: here I’m looking for conceptual stories.

^{^}

Here we might try to talk about the difference between de dicto vs. de re approaches to normative deliberation, but I don’t actually think this covers at it, as I think the thing I want to point at extends to nihilism worlds as well.

^{^}

Though if they’re too difficult – and especially conditional on anti-realism — one wonders whether the right sort of “should” is at stake. See Soares on “‘Should’ considered harmful.”

^{^}

And this despite the fact that the true meta-ethics is probably necessarily true.

^{^}

I even feel something similar towards the paperclipper who says, in that charming robot voice, “obviously, if there were a righteous God, we should worship.”

^{^}

I think my treatment of this topic in the context of 1 and 2 may end up inconsistent, but I’m going to leave it for now.

^{^}

Thanks to Katja Grace for discussion of this approach.

^{^}

See Nagel’s “Concealment and Exposure” for some interesting discussion of dynamics in this vicinity.

^{^}

Or at least, it used to? How far hath we fallen?

^{^}

Though importantly, norm-violation can be motivated by more than selfishness.

^{^}

Thanks to Katja Grace for suggesting this frame.

^{^}

In fact, it makes me want to say: "wake up! You live in the real world. Your actions have real consequences."

^{^}

See Dover (2019) for more on this and related problems for hypocrisy norms.

^{^}

Thanks to Claire Zabel for discussion.

^{^}

Though: careful now. If you aren’t a good person, that likely has something importantly to do with your relationship to “the point.”

^{^}

See Soares on “How we will be measured” for more.

^{^}

See e.g. Soares on excuses.

^{^}

From Kelsey Piper, on Don’t Look Up: “It’s the most serious thing the movie has to say… Don’t Look Up isn’t about ordinary people who discover inside them the heroism to save the ones they love. It’s about ordinary people who know what’s coming and aren’t ultimately heroes at all. They make a couple of futile attempts to do something, which amount to nothing. And then they die, because that’s what will happen if we aren’t up to the task ahead of us.” See also Robert Miles on “There’s No Rule That Says We Make It.”

^{^}

I think various spiritual traditions may be partly to blame, here. Sometimes, in Buddhism (and especially re: Karma), there’s a vibe like “get the intention right, and the rest will follow.” New agers, too, love to go off of the “energy” of something. And in Christianity, it can seem like one is expected, always, to know in one’s heart what is right and what is wrong, and the question is only: whether you have the inner strength to choose the right. Indeed, in my opinion, much of today’s moral discourse assumes much too hard that various moral questions are easy, and that the truth of blah position is clear to anyone who isn’t obviously an asshole, and that whether you are an asshole is quite immediately epistemically accessible if you would just, for a moment, stop to check. And perhaps, in many moral contexts, this isn’t too bad a guide. But I worry it misses the sense in which being a force for good in the world is not, centrally, a matter of introspection.

LESSWRONG
LW

LESSWRONG
LW

76

76

76

1. Introduction

2. Initial gestures

2.1 Some words

2.2 Some feelings

3. Like truth-seeking, but for agency as a whole

3.1 “Should I do X?” – fake and real

3.2 The proper object of agency

3.3 Is sincerity the opposite of weakness of will? (No.)

3.4 Fake weakness of will

4. Not playing pretend

5. Healthy stapleclippers

6. Seriousness

7. Other associations with non-ego/altruism/goodness

7.1 How close is the connection between sincerity and goodness?

8. Can we unify all these associations under a single principle?

8.1 True paths

8.1.1 If there were a worthy and to-be-served God, would you want to serve Him?

8.2 What type of not-playing-pretend does the social world care about?

8.3 Living with disunity

9. Sincerity’s failure modes