New Comment
Rendering 0/194 comments, sorted by
top scoring
(show all) Click to highlight new comments since: Moderation Log
Curated and popular this week
The complaints I remember about this post seem mostly to be objecting to how some phrases were distilled into the opening short "guideline" section. When I go reread the details it mostly seems fine. I have suggestions on how to tweak it.
(I vaguely expect this post to get downvotes that are some kind of proxy for vague social conflict with Duncan, and I hope people will actually read what's written here and vote on the object level. I also encourage more people to write up versions of The Basics of Rationalist Discourse as they seem them)
The things I'd want to change are:
1. Make some minor adjustments to the "Hold yourself to the absolute highest standard when directly modeling or assessing others' internal states, values, and thought processes." (Mostly, I think the word "absolute" is just overstating it. "Hold yourself to a higher standard" seems fine to me. How much higher-a-standard depends on context)
2. Somehow resolve an actual confusion I have with the "...and behave as if your interlocutors are also aiming for convergence on truth" clause. I think this is doing important, useful work, but a) it depends on the situation, b) it feels like it's not quite stating the right thing.
Digging into #2...
Okay, so when I reread the detailed section, I think I basically don't object to anything. I think the distillation sentence in the opening paragraphs conveys a thing that a) oversimplifies, and b) some people have a particularly triggered reaction to.
The good things this is aiming for that I'm tracking:
I think maybe the thing I want to replace this with is more like "aim for about 1-2 levels more trusting-that-everyone-is-aiming-for-truth than currently feel warranted, to account for your own biases, and to lead by example in having the conversation focus on truth." But I'm not sure if this is quite right either.
...
This post came a few months before we created our New User Reject Template system. It should have at least occurred to me to use some of the items here as some of the advice we have easily-on-hand to give to new users (either as part of a rejection notice, or just "hey, welcome to LW but it seems like you're missing some of the culture here."
If this post was voted in the Top 50, and a couple points were resolved, I'd feel good making a making a fork with minor context-setting adjustments and then linking to it as a moderation resource), since I'd feel like The People had a chance to weigh in on it.
The context-setting I'm imagining is not "these are the official norms of LessWrong", but, if I think a user is making a conversation worse for reasons covered in this post, be more ready to link to this post. Since this post came out, we've developed better Moderator UI for sending users comments on their comments, and it hadn't occurred to me until now to use this post as reference for some of our Stock Replies.
(Note: I currently plan to make it so, during the Review, anyone write Reviews on a post even if normally blocked on commenting. Ideally I'd make it so they can also comment on Review comments. I haven't shipped this feature yet but hopefully will soon)
1
Introduction
This post is meant to be a linkable resource. Its core is a short list of guidelines (you can link directly to the list) that are intended to be fairly straightforward and uncontroversial, for the purpose of nurturing and strengthening a culture of clear thinking, clear communication, and collaborative truth-seeking.
There is also (for those who want to read more than the simple list) substantial expansion/clarification of each specific guideline, along with justification for the overall philosophy behind the set.
Prelude: On Shorthand
Once someone has a deep, rich understanding of a complex topic, they are often able to refer to that topic with short, simple sentences that correctly convey the intended meaning to other people with similar context and expertise.
However, those same short, simple sentences are often dangerously misleading, in the hands of a novice who lacks the proper background. Dangerous precisely because they seem straightforward and comprehensible, and thus the novice will confidently extrapolate outward from them in what feel like perfectly reasonable ways, unaware the whole time that the concept in their head bears little or no resemblance to the concept that lives in the expert's head.
Good shorthand in the hands of an experienced user need only be an accurate fit for the already-existing concept it refers to—it doesn't need the additional property of being an unmistakeable non-fit for other nearby attractors. It doesn't need to contain complexity or nuance—it just needs to remind the listener of the complexity already contained in their mental model. It's doing its job if it efficiently evokes the understanding that already exists, independent of itself.
This is important, because what follows this introduction is a list of short, simple sentences comprising the basics of rationalist discourse. Each of those sentences is a solid fit for the more-complicated concept it's gesturing at, provided you already understand that concept. The short sentences are mnemonics, reminders, hyperlinks.
They are not sufficient, on their own, to reliably cause a beginner to construct the proper concepts from the ground up, and they do not, by themselves, rule out all likely misunderstandings.
All things considered, it seems good to have a clear, concise list near the top of a post like this. People should not have to scroll and scroll and sift through thousands of words when trying to refer back to these guidelines.
But each of the short, simple sentences below admits of multiple interpretations, some of which are intended and others of which are not. They are compressions of complex points, and compressions are inevitably lossy. If a given guideline is new to you, check the in-depth explanation before reposing confidence in your understanding. And if a given guideline stated-in-brief seems to you to be flawed or misguided in some obvious way, check the expansion before spending a bunch of time marshaling objections that may well have already been answered.
Further musing on this concept: Sazen
Guidelines, in brief:
0. Expect good discourse to require energy.
What does it mean for something to be a "guideline"?
Think of the above, then, as a set of priors. If a guideline says "Do [X]," that is intended to convey that:
Thus, given the goals of clear thinking, clear communication, and collaborative truth-seeking, the burden of proof is on a given guideline violation to justify itself. There will be many cases in which violating a guideline will in fact be exactly the right call, just as the next marble drawn blindly from a bag of mostly red marbles may nevertheless be green. But if you're doing something that's actively contra to one of the above, it should be for a specific, known reason, that you should be willing to discuss if asked (assuming you didn't already explain up front).
Which leads us to the Zeroth Guideline: expect good discourse to (sometimes) require energy.
If it did not—if good discourse were a natural consequence of people following ordinary incentives and doing what they do by default—then it wouldn't be recognizable as the separate category of good discourse.
A culture of (unusually) clear thinking, (unusually) clear communication, and (unusually) collaborative truth-seeking is not the natural, default state of affairs. It's endothermic, requiring a regular influx of attention and effort to keep it from degrading back into a state more typical of the rest of the internet.
This doesn't mean that commentary must always be high effort. Nor does it mean that any individual user is on the hook for doing a hard thing at any given moment.
But it does mean that, in the moments where meeting the standards outlined above would take too much energy (as opposed to being locally unnecessary for some other, more fundamental reason), one should lean toward saying nothing, rather than actively eroding them.
Put another way: a frequent refrain is "well, if I have to put forth that much effort, I'll never say anything at all," to which the response is often "correct, thank you."
It's analogous to a customer complaining "if Costco is going to require masks, then I'm boycotting Costco." All else being equal, it would be nice for customers to not have to wear masks, and all else being equal, it would be nice to lower the barrier to communication such that more thoughts could be more easily included.
But all else is not equal; there are large swaths of common human behavior that are corrosive or destructive to the collaborative search for truth. No single contributor or contribution is worth sacrificing the overall structures which allow for high-quality conversation in the first place—if one genuinely does not have the energy required to e.g. put forth one's thoughts while avoiding straightforwardly false statements, or while distinguishing inference from observation (etc.), then one should simply disengage.
Note that there is always room for discussion on the meta level; it is not the case that there is universal consensus on every norm, nor on how each norm looks in practice (though the above list is trying pretty hard to limit itself to norms that are on firm footing).
Note also that there is a crucial distinction between [fake/performative verbal gymnastics], and [sincere prioritization of truth and accuracy]—more on this in Sapir-Whorf for Rationalists.
For most practical purposes, this is the end of the post. All remaining sections are reference material, meant to be dug into only when there's a specific reason to; if you read further, please know that you are doing the equivalent of reading dictionary entries or encyclopedia entries and that the remaining words are not optimized for being Generically Entertaining To Consume.
Where did these come from?
I tinkered with drafts of this essay for over a year, trying to tease out something like an a priori list of good discourse norms, and wrestling with various imagined subsets of the LessWrong audience and trying to predict what objections might arise, and the whole thing was fairly sprawling and I ultimately scrapped it in favor of just making a list of a dozen in-my-estimation unusually good rationalist communicators, and then writing down the things that made those people's discourse stand out to me in the first place, i.e. the things it seems to me that they do a) 10-1000x more frequently than genpop, and b) 2-10x more frequently than the median LessWrong user.
That list comprised:
I claim that if you contrast the words produced by the above individuals with the words produced by the rest of the English-speaking population, what you find is approximately the above ten guidelines.
In other words, the guidelines are descriptive of good discourse that already exists; here I am attempting to convert them into prescriptions, with some wiggle room and some caveats. But they weren't made up from whole cloth; they are in fact an observable part of What Actually Works In Practice.
Some of the above individuals have specific deficits in one or two places, perhaps, and there are some additional things that these individuals are doing which are not basic, and not found above. But overall, the above is a solid 80/20 on How To Talk Like Those People Do, and sliding in that direction is going to be good for most of us.
Why does this matter?
In short: because the little things add up. For more on this, take a look at Draining the Swamp as an excellent metaphor for how ambient hygiene influences overall health, or revisit Concentration of Force, in which I lay out my argument for why we should care about small deltas on second-to-second scales, or Moderating LessWrong, which is sort of a spiritual precursor to this post.
Expansions
1. Don't say straightforwardly false things.
... and be ready and willing to explicitly walk back unintentional falsehoods, if asked or if it seems like it would help your conversational partner.
In normal social contexts, where few people are attending to or attempting to express precise truth, it's relatively costless to do things like:
Most of the times that people end up saying straightforwardly false things, they are not intending to lie or deceive, but rather following one of these incentives (or similar).
However, if you are actively intending to create, support, and participate in a culture of clear thinking, clear communication, and collaborative truth-seeking, it becomes more important than usual to break out of those default patterns, as well as to pump against other sources of unintentional falsehood like the typical mind fallacy.
This becomes even more important when you consider that places like LessWrong are cultural crossroads—users come from a wide variety of cultures and cannot rely on other users sharing the same background assumptions or norms-of-speech. It's necessary in such a multicultural environment to be slower, more careful, and more explicit, if one wants to avoid translation errors and illusions of transparency and various other traps and pitfalls.
Some ways you might feel when you're about to break the First Guideline:
Some ways a First Guideline request might look:
2. Track and distinguish your inferences from your observations.
... or be ready and willing to do so, if asked or if it seems like it would help your conversational partner (or the audience). i.e. build the habit of tracking the distinction between what something looks like, and what it definitely is.
The first and most fundamental question of rationality is "what do you think you know, and why do you think you know it?"
Many people struggle with this question. Many people are unaware of the processing that goes on in their brains, under the hood and in the blink of an eye. They see a fish, and gloss over the part where they saw various patches of shifting light and pattern-matched those patches to their preexisting concept of "fish." Less trivially, they think that they straightforwardly observe things like:
... and they miss the fact that they were running a bunch of direct sensory data through a series of filters and interpreters that brought all sorts of other knowledge and assumptions and past experience and causal models into play. The process is so easy and so habitual that they do not notice it is occurring at all.
(Where "they" is also meant to include "me" and "you," at least some of the time.)
Practice the skill of slowing down, and zooming in. Practice asking yourself "why?" after the fashion of a curious toddler. Practice answering the question "okay, but if there were another step hiding in between these two, what would it be?" Practice noticing even extremely basic assumptions that seem like they never need to be stated, such as "Oh! Right. I see the disconnect—the reason I think X is worse than Y is because as far as I can tell X causes more suffering than Y, and I think that suffering is bad."
This is particularly useful because different humans reason differently, and that reasoning tends to be fairly opaque, and attempting to work backward from [someone else's outputs] to [the sort of inputs you would have needed, to output something similar] is a recipe for large misunderstandings.
Wherever possible, try to make explicit the causes of your beliefs, and to seek the causes underlying the beliefs of others, especially when you strongly disagree. Work on improving your ability to tease out what you observed separate from what you interpreted it to mean, so that the conversation can track (e.g. "I saw A," "I think A implies B," and "I don't like B" as three separate objects. If you're unable to do so, for instance because you do not yet know the source of your intuition, try to note out loud that that's what's happening.
Some ways you might feel when you're about to break the Second Guideline:
Some ways a Second Guideline request might look:
3. Estimate and make clear your rough level of confidence in your assertions.
... or be ready and willing to do so, if asked or if it seems like it would help another user.
Humans are notoriously overconfident in their beliefs, and furthermore, most human societies reward people for visibly signaling confidence.
Humans, in general, are meaningfully influenced by confidence/emphasis alone, separate from truth—probably not literally all humans all of the time, but at least in expectation and in the aggregate, either for a given individual across repeated exposures or for groups of individuals (more on this in Overconfidence is Deceit).
Humans are social creatures who tend to be susceptible to things like halo effects, when not actively taking steps to defend against them, and who frequently delegate and defer and adopt others' beliefs as their own tentative positions, pending investigation, especially if those others seem competent and confident and intelligent. If you expose 1000 randomly-selected humans to a debate between a quiet, reserved person outlining an objectively correct position and a confident, emphatic person insisting on an unfounded position, many in that audience will be net persuaded by the latter, and others will feel substantially more uncertainty and internal conflict than the plain facts of the matter would have left them feeling by default.
Thus, there is frequently an incentive to misrepresent your confidence, for instrumental advantage, at the cost of our collective ability to think clearly, communicate clearly, and engage in collaborative truth-seeking.
Additionally, there is a tendency among humans to use vague and ambiguous language that is equally compatible with multiple interpretations, such as the time that a group converged on agreement that there was "a very real chance" of a certain outcome, only to discover later, in one-on-one interviews, that at least one person meant that outcome was 20% likely, and at least one other meant it was 80% likely (which are exactly opposite claims, in that 20% likely means 80% unlikely).
Thus, it behooves people who want to engage in and encourage better discourse to be specific and explicit about their confidence (i.e. to use numbers, and to calibrate your use of numbers over time, or to flag tentative beliefs as tentative, or to be clear about the source of your belief and your credence in that source).
Some ways you might feel when you're about to break the Third Guideline:
Some ways a Third Guideline request might look:
4. Make your claims clear, explicit, and falsifiable, or explicitly acknowledge that you aren't doing so (or can't).
... or at least be ready and willing to do so, if asked or if it seems like it would help make things more comprehensible.
It is, in fact, actually fine to be unsure, or to have a vague intuition, or to make an assertion without being able to provide cruxes or demonstrate how it could be proven/disproven. None of these things are disallowed in rational discourse.
But noting aloud that you are self-aware about the incomplete nature of your argument is a highly valuable social maneuver. It signals to your conversational partner "I am aware that there are flaws in what I am saying; I will not take it personally if you point at them and talk about them; I am taking my own position as object rather than being subject to it and tunnel-visioned on it."
(This is a move that makes no sense in an antagonistic, zero-sum context, since you're just opening yourself up to attack. But in a culture of clear thinking, clear communication, and collaborative truth-seeking, contributing your incomplete fragment of information, along with signaling that yes, the fragment is, indeed, a fragment, can be super productive.)
Much as we might wish that everyone could take for granted that disagreement is prosocial and productive and not an attack, it is not actually the case. Some people do indeed launch attacks under the guise of disagreement; some people do indeed respond to disagreement as if it were an attack even if it is meant entirely cooperatively; some people, fearing such a reaction, will be hesitant to note their disagreement in the first place, especially if their conversational partner doesn't seem open to it.
The more clear it is what, exactly, you're trying to say, the easier it is for other people to evaluate those claims, or to bring other information that's relevant to the issue at hand.
The more your assertions manage to be checkable, the easier it is for others to trust that you're not simply throwing spaghetti at the wall to see what sticks.
And the more you're willing to flag your own commentary when it fails on either of the above, the easier it is to contribute to and strengthen norms of good discourse even with what would otherwise be a counterexample. Pointing out "this isn't great, but it's the best that I've got" lets you contribute what you do have, without undermining the common standard of adequacy.
Some ways you might feel when you're about to break the Fourth Guideline:
Some ways a Fourth Guideline request might look:
5. Aim for convergence on truth, and behave as if your interlocutors are also aiming for convergence on truth.
... and be ready to falsify your impression otherwise, if evidence starts to pile up.
The goal of rationalist discourse is to be less wrong—for each of us as individuals and all of us as a group to have more correct beliefs, and fewer incorrect beliefs.
If two people disagree, it's tempting for them to attempt to converge with each other, but in fact the right move is for both of them to try to see more of what's true.
If you are moving closer to truth—if you are seeking available information and updating on it to the best of your ability—then you will inevitably eventually move closer and closer to agreement with all the other agents who are also seeking truth.
However, when conversations get heated—when the stakes are high—when the other person not only appears to be wrong but also to be acting in poor faith—that's when it's the most valuable to keep in touch with the possibility that you might be misunderstanding each other, or that the problem might be in your models, or that there might be some simple cultural or norms mismatch, or that your conversational partner might simply be locally failing to live up to standards that they do, in fact, generally hold dear, etc.
It's very easy to observe another person's output, evaluate it according to your own norms and standards, and conclude that you understand their motives and that those motives are bad.
It is not, in fact, the case that everyone you engage with is primarily motivated by truth-seeking! Even in enclaves like LessWrong, there are lots of people who are prioritizing other goals over that one a substantial chunk of the time.
But simple misunderstandings, and small, forgivable, recoverable slips in mood or mental discipline outnumber genuine bad faith by a large amount. If you are running a tit-for-tat algorithm in which you quickly respond to poor behavior by mirroring it back, you will frequently escalate a bad situation (and often appear, to the other person, like the first one who broke cooperation).
Another way to think of this is: it pays to give people two extra chances to demonstrate that they are present in good faith and genuinely trying to cooperate, because if they aren't, they'll usually prove it soon enough anyway. You don't have to turn the other cheek repeatedly, but doing so once or twice more than you would by default goes a long way toward protecting against false positives on your bad-faith detector.
This behavior can be modeled, as well—the quickest way to get a half-derailed conversation back on track is to start sharing pairs of [what you believe] and [why you believe it]. To demonstrate to your conversational partner that those two things go together, and show them the kind of conversation you want to have.
(This is especially useful on the meta level—if you are frustrated, it's much better to say "I'm seeing X, and interpreting it as meaning Y, and feeling frustrated about that!" than to just say "you're being infuriating.")
You could think of the conversational environment as one in which defection strategies are rampant, and many would-be cooperators have been trained and traumatized into hair-trigger defection by repeated sad experience.
Taking that fact into account, it's worth asking "okay, how could I behave in such a way as to invite would-be cooperators who are hair-trigger defecting back into a cooperative mode? How could I demonstrate to them, via my own behavior, that it's actually correct to treat me as a collaborative truth-seeker, and not as someone who will stab them as soon as I have the slightest pretext for writing them off?"
Some ways you might feel when you're about to break the Fifth Guideline:
Some ways a Fifth Guideline request might look:
6. Don't jump to conclusions—maintain at least two hypotheses consistent with the available information.
... or be ready and willing to generate a real alternative to your main hypothesis, if asked or if it seems like it would help another user.
There exists a full essay on this concept titled Split and Commit. The short version is that there is a large difference between a person who has a single theory (which they are nominally willing to concede might be false), and a person who has two fully distinct possible explanations for their observations, and is looking for evidence to distinguish between them.
Another way to point at this distinction is to remember that bets are different from beliefs.
Most of the time, you are forced to make some sort of implicit bet. For instance, you have to choose how to respond to your conversational partner, and responding-to-them-as-if-they-were-sincere is a different "bet" than responding-to-them-as-if-they-are-insincere.
And because people are often converting their beliefs into bets, and because bets are often effectively binary, they often lose track of the more complicated thing that preceded the rounding-off.
If a bag of 100 marbles contains 70 red ones and 30 green ones, the best bet for the first string of ten marbles out of the bag is RRRRRRRRRR. Any attempt to sprinkle some Gs into your prediction is more likely to be wrong than right, since any single position is 70% likely to contain an R.
(There's less than a 3% chance of the string being RRRRRRRRRR, but the odds of any other specific string are even worse.)
But it would be silly to say that you believe that the next ten marbles out of the bag will all be red. If forced, you will predict RRRRRRRRRR, because that's the least wrong prediction, but actually (hopefully) your belief is "for each marble, it's more likely to be red than green but it could pretty easily be green."
In similar fashion, when you witness someone's behavior, and your best bet is "this person is biased or has an unstated agenda," your belief should ideally be something like "this behavior is most easily explained by an unstated agenda, but if I magically knew for a fact that that wasn't what was happening, the next most likely explanation would be ______________."
That extra step—of pausing to consider what else might explain your observations, besides your primary theory—is one that is extremely useful, and worth practicing until it becomes routine. People who do not have this reflex tend to fall into many more pits/blindspots, and to have a much harder time bridging inferential gaps, especially with those they do not already agree with.
Some ways you might feel when you're about to break the Sixth Guideline:
Some ways a Sixth Guideline request might look:
7. Be careful with extrapolation, interpretation, and summary/restatement.
Distinguish between what was actually said and what it sounds like/what it implies/what you think it looks like in practice/what it's tantamount to, especially if another user asks you to pay more attention to this distinction than you were doing by default. If you believe that a statement A strongly implies B, and you are disagreeing with A because you disagree with B, explicitly note that "A strongly implies B" is a part of your model. Be willing to do these things on request if another person asks you to, or if you notice that it will help move the conversation in a healthier direction.
Another way to put this guideline is "don't strawman," but it's important to note that, from the inside, strawmanning doesn't typically feel like strawmanning.
"Strawmanning" is a term for situations in which:
Person B constructs a strawman, in other words, just so they can then knock it down.
...
There's a problem with the definition above; readers are invited to pause and see if they can catch it.
...
If you'd like a hint: it's in the last line (the one beginning with "Person B constructs a strawman").
...
The problem is in the last clause.
"Just so they can knock it down" presupposes purpose. Not only is Person B engaging in misrepresentation, they're doing it in order to have some particular effect on the larger conflict, presumably in the eyes of an audience (since knocking over a strawman won't do much to influence Person A).
It's a conjunction of act and intent, implying that the vast majority of people engaging in strawmanning are doing so consciously, strategically, and in a knowingly disingenuous fashion—or, if they're not fully self-aware about it, they're nevertheless subconsciously optimizing for making Person A's position appear sillier or flimsier than it actually is.
This does not match how the term is used, out in the wild; it would be difficult to believe that even twenty percent of my own encounters with others using the term (let alone a majority, let alone all of them) are downstream of someone being purposefully deceptive. Instead, the strawmanning usually seems to be "genuine," in that the other person really thinks that the position being argued actually is that dumb/bad/extreme.
It's an artifact of blind spots and color blindness; of people being unable-in-practice to distinguish B from A, and therefore thinking that A is B, and not realizing that "A implies B" is a step that they've taken inside their heads. Different people find different implications to be more or less "obvious," given their own cultural background and unique experiences, and it's easy to typical-mind that the other relevant people in the conversation have approximately the same causal models/context/knowledge/anticipations.
If it's just patently obvious to you that A strongly implies B, and someone else says A, it's very easy to assume that everyone else made the leap to B right along with you, and that the author intended that leap as well (or intended to hide it behind the technicality of not having actually come out and said it). It may feel extraneous or trivial, in the moment, to make that inference explicit—you can just push back on B, right?
Indeed, if the leap from A to B feels obvious enough, you may literally not even notice that you're making it. From the inside, a blindspot doesn't feel like a blindspot—you may have cup-stacked your way straight from A to B so quickly and effortlessly that your internal experience was that of hearing them say B, meaning that you will feel bewildered yourself when what seems to you to be a perfectly on-topic reply is responded to as though it were an adversarial non-sequitur.
(Which makes you feel as if they broke cooperation first; see the sixth guideline.)
People do, in fact, intend to imply things with their statements. People's sentences are not contextless objects of unambiguous meanings. It's entirely fine to hazard a guess as to someone's intended implications, or to talk about what most people would interpret a given sentence to mean, or to state that [what they wrote] landed with you as meaning [something else]. The point is not to pretend that all communication is clear and explicit; it's to stay in contact with the inherent uncertainty in our reconstructions and extrapolations.
"What this looks like, in practice" or "what most people mean by statements of this form" are conversations that are often skipped over, in which unanimous consensus is (erroneously) taken for granted, to everyone's detriment. A culture that seeks to promote clear thinking, clear communication, and collaborative truth-seeking benefits from a high percentage of people who are willing to slow down and make each step explicit, thereby figuring out where exactly shared understanding broke down.
Some ways you might feel when you're about to break the Seventh Guideline:
Some ways a Seventh Guideline request might look:
8. Allow people to restate, clarify, retract, and redraft their points.
Communication is difficult. Good communication is often quite difficult.
One of the simplest interventions for improving discourse is to allow people to try again.
Sometimes our first drafts are clumsy in their own right—we spoke too soon, or didn't think things through deeply enough.
Other times, we said words which would have caused a clone of ourselves to understand, but we failed to account for some crucial cultural difference or inferential gap with our non-clone audience, and our words caused them to construct a meaning that was very different than the meaning we intended.
Also, sometimes we're just wrong!
It's quite common, on the broader internet and in difficult in-person conversations, for people's early rough-draft attempts to convey a thought to haunt them. People will relentlessly harp on some early, clumsy phrasing, or act as if some point with unpleasant ramifications (which the speaker failed to consider) intended those ramifications.
What this results in is a chilling effect on speech (since you feel like you have to get everything right on your first try or face social punishment) and a disincentive for making updates and corrections (since those corrections will often simply be ignored and you'll be punished anyway as if you never made them, so why bother).
Part of the solution is to establish a culture of being forgiving of imperfect first drafts (and generous/light-touch in your interpretation of them), and of being open to walkbacks or restatements or clarifications.
It's perfectly acceptable to say something like "This sounds crazy/abhorrent/wrong to me," or to note that what they wrote seems to you to imply some statement B that is bad in some way.
It's also perfectly reasonable to ask that people demonstrate that they see what was wrong with their first draft, rather than just being able to say "no, I meant something subtly different" ad infinitum.
But if your conversational partner replies with "oh, gosh, sorry, no, that is not what I'm trying to say," it's generally best to take that assertion at face value, and let them start over. As with the sixth guideline, this means that you will indeed sometimes be giving extra leeway to people who are actually being irrational/unreasonable/bad/wrong, but most of the time, it means that you will be avoiding the failure mode of immediately leaping to a conclusion about what the other person meant and then refusing to relinquish that assumption.
The claim is that the costs of letting a few more people "get away with it" a little longer is better than curtailing the whole population's ability to think out loud and update on the fly.
Some ways you might feel when you're about to break the Eighth Guideline:
Some ways an Eighth Guideline request might look:
9. Don't weaponize equivocation/abuse categories/engage in motte-and-bailey shenanigans.
...and be open to eschewing/tabooing broad summary words and talking more about the details of your model, if another user asks for it or if you suspect it would lower the overall confusion in a given interaction.
Labels are great.
However, labels are a tool with some known failure modes. When someone uses a conceptual handle like "marriage," "genocide," "fallacy of the grey," or "racist," they are staking a claim about the relationship between a specific instance of [a thing in reality], and a cluster of [other things] that all share some similar traits.
That leads to some fairly predictable misunderstandings.
For instance, someone might notice that a situation has (e.g.) three out of seven salient markers of gaslighting (in their own personal understanding of gaslighting).
Three out of seven is a lot, when most things have zero out of seven! So it's reasonable for them to bring in the conceptual handle "gaslighting" as they begin to reason about and talk about the situation—to port in the intuitions and strategies that are generally useful for things in the category.
But it's very easy for people to fail to make clear that they're using the term "gaslighting" because it had specific markers X, Y, and Z, and that the situation doesn't seem to have markers T, U, V, or W at all, let alone considerations of whether or not their own idiosyncratic seven markers sync up with consensus understanding of gaslighting.
And thus the application of the term can easily cause other observers to implicitly conclude that all of T, U, V, W, X, Y, and Z are nonzero involved (and possibly also Q, R, and S that various other people bring to the table without realizing that they are non-universal).
Done intentionally, we call this weaponized equivocation or motte-and-bailey, i.e. "I can make the term gaslighting stick in a technically justified sense, and then abuse the connotation to make everybody think that you were doing all of the bad things involved in gaslighting on purpose and that you are a gaslighter, with all that entails."
But it also happens by accident, quite a lot. A conceptual handle makes sense to Person A, so they use it, and Person B both loses track of nuance and also injects additional preconceptions, based on their understanding of the conceptual handle.
The general prescription is to use categories and conceptual handles as a starting point, and then carefully check one's understanding.
Another way to think of this prescription is to recognize that the use of categories and conceptual handles is warping, in the sense that categories and conceptual handles are often like gravitational attractors pulling people's models toward a baseline archetype or stereotype. They tend to loom large, and obscure away detail, and generate a kind of top-down smoothing consensus or simplification.
That's super useful when the alternative is having them be lost out in deep space, but it's also not as good as using the category to get them in the right general vicinity and then deliberately not leaning on the category once they're close enough that you can talk about all of the relevant specifics in detail.
Some ways you might feel when you're about to break the Ninth Guideline:
Some ways a Ninth Guideline request might look:
10. Hold yourself to the absolute highest standard when directly modeling or assessing others' internal states, values, and thought processes.
Of the ten guidelines, this is the one which is the least about epistemic hygiene, and the most about social dynamics.
(It's not zero about epistemic hygiene, but it deserves extra emphasis for pragmatic reasons rather than philosophical ones.)
In short:
If you believe that someone is being disingenuous or crazy or is in the grips of a blindspot—if you believe that you know, better than they know themselves, what's going on in their head (or perhaps that they are lying about what's going on in their head)—then it is important to be extra cautious and principled about how you go about discussing this fact.
This is important because it's very easy for people to (reasonably) feel attacked or threatened or delegitimized when others are making bold or judgment-laden assertions about the internal contents of their mind/soul/values, and it's very hard for conversation to continue to be productive when one of the central participants is partially or fully focused on defending themselves from perceived social attack.
It is actually the case that people are sometimes crazy. It is actually the case that people are sometimes lying. It is actually the case that people are sometimes mistaken about the contents of their own minds, and that other people, on the outside, can see this more clearly. A blanket ban on hypotheses-about-others'-internals would be crippling to anyone trying to see clearly and understand the world; these things should, indeed, be thinkable and discussible, the fact that they are "rude" notwithstanding.
But by making those hypotheses a part of an open conversation, you're adding a great deal of social and emotional strain to the already-difficult task of collaborative truth-seeking with a plausibly-compromised partner. In many milieus, the airing of such a hypothesis is an attack; there are not a lot of places where "you might be crazy" or "I know more than you about how your mind works" is a neutral or prosocial move. If the situation is such that it feels genuinely crucial for you to raise such a hypothesis out loud, then it should also be worth correspondingly greater effort and care.
(See the zeroth guideline.)
Some simple actions that tend to make this sort of thing go less reliably badly:
For more on this, see split and commit.
Some ways you might feel when you're about to break the Tenth Guideline:
Some ways a Tenth Guideline request might look:
(These requests deliberately written to appear somewhat triggered/hostile because that's the usual tone by the point such a request needs to be made, and a little bit of leeway on behalf of the beleaguered seems appropriate.)
Appendix: Miscellaneous Thoughts
This post was long, and was written over the course of many, many months. Below are some scattered, contextless snippets of thought that ended up not having a home in any of the sections above.
Some general red flags for poor discourse:
Some sketchy conversational movements that don't fall neatly into the above:
A skill not mentioned elsewhere in this post: the meta-skill of being willing to recognize, own up to, apologize for, and correct failings in any of the above, rather than hiding one's shame or doubling down or otherwise acting as if the problem is the mistake being seen rather than the mistake being made.
Author's note: These days, my thoughts go onto my substack instead of onto LessWrong. Everything I write becomes free after a week or so, but it’s only paid subscriptions that make it possible for me to write. If you found a coffee’s worth of value in this or any of my other work, please consider signing up to support me; every bill I can pay with writing is a bill I don’t have to pay by doing other stuff instead. I also accept and greatly appreciate one-time donations of any size.
Appendix: Sabien's Sins
The following is something of a precursor to the above list of basics; it was not intended to be as complete or foundational as the ten presented here (this is more my own personal preferred norms, rather than an attempt to describe good discourse in a more neutral, general sense). It was posted on Facebook several years ago; I include it here mostly as a historical curiosity.