What epistemic hygiene norms should there be?


14


Kaj_Sotala

The wiki entry for Epistemic Hygiene defines the term as:

Epistemic hygiene consists of practices meant to allow accurate beliefs to spread within a community and keep less accurate or biased beliefs contained. The practices are meant to serve an analogous purpose to normal hygiene and sanitation in containing disease.

The term was coined in Steve Rayhawk's and Anna Salamon's post "The ethic of hand-washing and community epistemic practice", and there have been several mentions of it around the site. But what, exactly, might good epistemic hygiene norms be?

I'm especially interested in this question in the context of meetup groups. In Less Wrong NYC: Case Study of a Successful Rationalist Chapter, Cosmos writes:

Epistemic privilege and meme-sharing: The most powerful aspect of a group of rationalists is that you have an entire class of people whose reasoning you trust. Division of labor arises naturally as each member has different interests, they all pursue a variety of skills and areas of expertise, which they can then bring back to the group. Even the lowest-level rationalists in the group can rapidly upgrade themselves by adopting winning heuristics from other group members. I cannot overstate the power of epistemic privilege. We have rapidly spread knowledge about metabolism, exercise, neuroscience, meditation, hypnosis, several systems of therapy... and don't forget the Dark Arts.

This would imply that one way that a meetup group (or for that matter, any social group) could get really successful would be by adopting great epistemic hygiene norms. But unless everyone present has read most of the Sequences - which gets increasingly unlikely the more the group grows - even most LW meetups probably won't just spontaneously enforce such norms. Wouldn't it be great if there were an existing list of such norms, that each group could look through and then decide which ones they'd like to adopt?

Here are some possible epistemic hygiene norms that I could find / come up with:

  • Be honest about your evidence and about the actual causes of your beliefs. Valuable for distinguishing accurate from mistaken beliefs. (The ethic of hand-washing)
  • Focus attention on the evidence and on the actual causes of both your beliefs, and beliefs of the people you're talking with. (The ethic of hand-washing)
  • Only pass on ideas that you've verified yourself. (Problematic, since any given individual can only verify a tiny fraction of all of their beliefs.) (The ethic of hand-washing)
  • Explicitly separate “individual impressions” (impressions based only on evidence you've verified yourself) from “beliefs” (which include evidence from others’ impressions). (Naming beliefs)
  • Give status for reasoned opinion-change in the face of good evidence, rather than considering the "loser" of a debate low-status. (The ethic of hand-washing)
  • Leave the other person a line of retreat in all directions, avoiding pressures that might wedge them towards either your ideas or their own. (The ethic of hand-washing)
  • Encourage people to present the strongest cases they can against their own ideas. (comment, Carl Shulman)
  • Encourage "Why do I think that" monologues. You elaborate on a thing you currently believe to be true by specifying the reasons you believe it, the reasons you believe the reasons, etc and trying to dig out the whole epistemological structure. (comment, JulianMorrison)
  • Find ways to track the sincerity and accuracy of what people have said in the past, and make such information widely available. (comment, mark_spottswood)
  • Don't trust evidence you don't remember the source of, even if you remember reading the primary source yourself. (Hygienic Anecdotes)
  • Be upfront about when you don't remember the source of your claim. (comment, PhilGoetz)
  • When asked a question, state the facts that led you to your conclusion, not the conclusion itself. (Just the facts, ma'am!)
  • If you basically agree with someone's argument, but want to point out a minor problem, start off your response with the words "I agree with your conclusion". (Support That Sounds Like Dissent)
  • When agreeing with someone's claim, distinguish between "I have independent evidence that should add to our confidence in the speaker's conclusion" and "based on the evidence others have presented, I now agree, but don't take my agreement as further reason to update". (comment, AnnaSalamon)
  • Don't judge people for having bad ideas, only judge the ideas. (Your Rationality is My Business)
  • Be careful when recommending books on sources where you are not an expert, particularly when they're on highly controversial topics and happen to support your own conclusions. (comment, Vladimir_M)
  • Before you stake your argument on a point, ask yourself in advance what you would say if that point were decisively refuted. If you wouldn't actually change your mind, search for a point that you find more convincing. (Is That Your True Rejection? at CATO Unbound)
  • In discussions, presume the kinds of conditions that are the least convenient for your argument. (The Least Convenient Possible World)
  • If people are trying to figure out the truth, don't mistake their opinions about facts for statements of values. (Levels of communication)

And of course, there's a long list of norms that basically amount to "don't be guilty of bias X", e.g. "avoid unnecessarily detailed stories about the future", "avoid fake explanations", "don't treat arguments as soldiers", etc.

Which of these norms do you consider the most valuable? Which seem questionable? Do you have any norms of your own to propose?