Crosspost from my blog.

Introduction

Last time, I asked about The problem of graceful deference. We have to defer to other people's judgements of fact and of value, because there are too many important questions to consider thoroughly ourselves. Is germline engineering moral? What should I work on to decrease existential risk? Do I really have to floss? Should I get vaccinated? How feasible is safe AGI? Pick one each month or year to start the long process of becoming an expert on; the rest you'll have to defer on, for now.

Deference leads to several important dangers. It causes information cascades and correlated failures; it creates false moral consensus and false impressions that a question is really settled; it cuts us off from powerful intrinsic motivation.

How can we defer in a way that harnesses the power of deference, while attenuating the dangers?

Below are some partial answers.

If you're interested in this topic, there's a lot more to be worked out, so you could take a crack at it. See the last section, "Open problems in graceful deference".

(Caveats: This is a list of tools, each of which you may or may not want to pick up and use. They are phrased as imperatives, but of course they are only good for some people in some contexts. You may feel uncomfortable with some of these recommendations—just remember that you're wrong, trust me we're all already deferring on most questions. So, we are ok-ish with how we are deferring—at least, ok-ish to the extent that we're already ok-ish right now in general; and thinking about how we are deferring could open up ways to make our situation better. These tools are for doing what we're already doing, but more gracefully. These tools are not for throwing out independent reason. And these tools aren't for, you know, feeling super guilty—I only use them a bit! But I usually like when I use them more.)

Extended table of contents

Here's a synopsis of the tools.

• When are you deferring?

  • Notice when you might be deferring. What does deferring feel like? How can you bring it to your conscious attention?
  • Just say that you might be deferring. No need to pretend you've already worked everything out yourself from first principles with your eyes closed, or to pretend you never have a stance about anything that you haven't worked out explicitly for yourself.

• When should you defer?

  • Factor out and fortify endorsed deference. You can speculate about big ideas without having to also necessarily call into question important parts of your life that should stay stable.
  • Respect the costs of independent investigation. Truly doubting something important is often very hard work, so it makes sense that you shouldn't necessarily always be doubting lots of important stuff.
  • Give stewardship, not authority. Even if you're deferring to someone, keep an eye out about whether and how you should continue to defer to them.

• Who are you deferring to?

  • Take me to your leader. If you feel an obligation to defend your belief in something, you can try just pointing to the person whose opinion you're deferring to, and hopefully they'll defend it for you.
  • Recount your sources. Say where you got information and ideas if you can.
  • Expose cruxes about deference. Just like it's often helpful to figure out what would change your mind about some concrete question, it's also helpful to figure out what would make you stop wanting to defer to someone about some topic.

• What are you deferring about?

  • Distinguish your independent components. If you have something to add beyond your deferential opinion, it's helpful to distinguish the part you're adding away from the rest of the opinion.
  • Retreat to non-deferential cruxes. If you were arguing using a strong claim that you have to defer about, you could try instead arguing using a claim that is weaker—so you don't have to defer about it—but still strong enough to carry your argument.

Notice when you might be deferring

Needless to say, it helps to be aware when you're deferring. A couple indicators (certainly not proof, but maybe cause to consider that you might be deferring):

• You directly associate an idea with a person.
• A claim "seems like it stands to reason" (or maybe it really does stand to reason), or "is something everyone knows", or is something you would just take for granted. You might e.g. be deferring to a summary given in a textbook, which is sort of true but doesn't give all the important detail and which you did not indepedently question. Or you might be deferring to judgements that are implied or suggested by other people's actions, even if not stated or argued for explicitly.
• You can't easily bring to mind the arguments or evidence for a judgement, or the next-level arguments for the arguments, or cruxes (observations that would change your mind), or specific doubts you have about the evidence, or a picture of what the alternative looks like. You might be deferring to your own cached thoughts, or to someone else's conclusions.
• You feel nervous to say the opposite—like you should hold your tongue, or like someone might get mad at you for saying something, or you might say something that you later seem dumb for having said. You might be deferring to others's moral judgements (or your imaginations of their judgements).
• You feel an experiential fringe of sanctimoniousness—like, "Ah, I see, you are not aware of this thing that the intelligentsia / elite / informed / experts / savvy people, let me help you out.". You feel comfortable to not worry too much that the newcomer's perspective will gain more cachet, and leave you working on something that few care about; you know that "the community" cares about the thing you're doing, and thinks it's important, but doesn't especially care about this thing that the newcomer is talking about. You might be deferring to the consensus of the group whose knowledge you are graciously sharing.

Just say that you might be deferring

If you realize that you have a bottom line judgement already, but that you HAVEN'T already really doubted and investigated the question, you can just say that. You can just say, "I feel pretty strongly that reprogenetics is a bad idea, though I won't argue for that position explicitly right now.". Or you can elaborate, e.g.: "I have a stance against reprogenetics, but I haven't thought about it much, so I might be wanting to go with the current generally accepted stance, or I might have an intuition that I haven't made explicit, or something.". Don't pretend that you are not deferring; don't pretend that you've investigated a bunch and come to an explicitly reasoned-out conclusion.

You can then go on to speculate about what your intuitive concerns might be about, who you are deferring to, why you want to defer to them, what your cruxes might be, what the reasons behind the consensus are, and so on. But by first acknowledging deference, you can go ahead with those speculations without feeling like they have to produce a justification for your bottom line judgement, or that you have to change your mind if you don't produce such a justification. You've already stated what your current judgement is, and you've already acknowledged that the source of your current judgement is likely to be mainly deference, not a concrete reason.

Allow others to defer, and to say they are deferring. Allow others to provisionally think through why the judgement is correct or incorrect without having to update their judgement just based on their own reasoning.

Factor out and fortify endorsed deference

In relation to his philosophical exercise of fundamentally doubting everything, Descartes writes in Discourse on Method, part three:

And finally, just as it is not enough, before beginning to rebuild the house where one is living, simply to pull it down, and to make provision for materials and architects or to train oneself in architecture, and also to have carefully drawn up the building plans for it; but it is also necessary to be provided with someplace else where one can live comfortably while working on it; so too, in order not to remain irresolute in my actions while reason required me to be so in my judgments, and in order not to cease to live as happily as possible during this time, I formulated a provisional code of morals, which consisted of but three or four maxims, which I very much want to share with you.

(https://grattoncourses.wordpress.com/wp-content/uploads/2017/12/rene-descartes-discourse-on-method-and-meditations-on-first-philosophy-4th-ed-hackett-pub-co-1998.pdf)

Separate out what you want to defer about from what you're going to really doubt. In particular, you're likely to want to mostly defer to society about which actions are generally very advisable or very inadvisable, even as you doubt the supposed justifications for those judgements.

That way you can more safely doubt some things without threatening your deference, or in other words, you can continue deferring while incurring less of a cost of restriction on what you can doubt. E.g. "Ok, I can discuss whether or not reprogenetics would hypothetically be good if safe and effective and accessible and legal and widely practiced, but either way I will not work on a project that's actually trying to do embryo editing.".

Respect the costs of independent investigation

Doubting something that's important to how you think and act is a fearsome undertaking. Respect the costs of doubting—i.e. the costs of maybe undoing some way that you have been deferring. Respect the costs of going against the grain, betting against the market, doing a bunch of cognitive labor yourself that you could have just copied from your society.

Because doubt is costly, it is dignified to defer instead. Defer with dignity. It is good to remember that, when you defer, you are drawing on the resources that your society provides to you. You could possibly have done more work on your own in order to produce better information and better judgement for yourself and for others; but it is respectable to choose which questions to struggle with and to mostly defer.

You don't have to come up with a reason for rejecting an idea that is not your true rejection. The most respectable thing is to do original work, solve a problem, and publicly demonstrate your solution; the least respectable thing is to defer and pretend that you are not deferring; and in the middle, respectable enough, is to defer and say you are deferring.

This is also something you can do to help others defer gracefully. If other people know that you understand that non-deference (independent investigation) is costly, then other people who are deferring can more comfortably just tell you "I'm deferring" rather than pretending to not defer.

Give stewardship, not authority

When you defer to someone, do not give them authority. Your judgements aren't their property to do with as they please.

They are the stewards of your judgements. You've given them concrete control, but that control is yours to modify or transfer or revoke, and you retain ultimate responsibility for your judgements.

Don't open niches for those who you defer-to, within which they can abuse their stewardship. Don't needlessly expand their control over your judgements. In other words, don't be a cult-follower towards anyone, even if they aren't yet being a cult-leader.

Keep accounts about whether the steward appropriately handled your judgements on your behalf. You can't necessarily hold them accountable, since your deference was a choice—you made that choice, you are responsible for it, and you may have made it without their input. But keep the accounts, usually ideally publicly.

Take me to your leader

There is a norm of debate. (For better or worse, it's a weak norm, deployed in few communities.) According to this norm, if you say X and Bob says not-X, then you should either debate Bob about X or else update to believe not-X.

This norm pressures people to not defer, because a judgement based on deference is not something you can stand up and defend with arguments and facts in a debate. Either you eject yourself from the communities that have that norm; or else you have to do a bunch of research and thinking in reaction to any random challenge; or else you have to fake having coherent positions in debates.

Instead of choosing one of those options, say: "I haven't investigated X deeply myself. What Carol says about X makes sense to me and I generally trust what she says about several topics. Further, so far she's successfully rebutted the critiques of her position. So, if you want to convince me about X, debate Carol and show her position about X to be wrong.".

And, to help others defer gracefully, treat that as a respectable response.

And, apply the debate-or-update norm more strongly to the leaders who are deferred-to. (Though this is fraught, if the leaders did not choose or make use of their position.)

Recount your sources

If you got an idea or insight or piece of information from Alice, and then you repeat it to Bob, also tell Bob that you got it from Alice.

People don't do this. Partly that's because it's hard to keep track and takes effort to recount in conversation. Partly it's because they want to sound smart—but that is a major transgression against God's will.

By recounting your sources to Bob, you let Bob know who to defer to, if he would want to defer to the source of what you shared with him.

If you fail to recount your sources, then you appear as though you aren't deferring, even when you are.

If you fail to recount your sources, then you open up your listeners to double-counting evidence, if they also hear other second-hand judgements that actually are transmissions from the same source as you are transmitting.

If you fail to recount your sources, then you make it harder for people to track and quarantine bad information. For example, if you say "NZT-48 causes brain bleeds.", what am I supposed to do with this information, if I know about the topic? Instead if you say "I read a study by Krombopulos et al. (2028) that says NZT-48 causes brain bleeds.", then I can be like "Yeah I've read that study, they totally screwed up their analysis, there's actually no effect.".

Expose cruxes about deference

Did COVID-19 originally leak from a lab? I don't even have a guess, but if I did, it would probably be based on deference to some expert in genetics and virology. You can't really argue to me about cleavage sites and base rates and so on (well you could, but it would take a lot of work). Would I then be, in practice, completely impervious to facts and reason? No, you could shake my judgement by convincing me that the expert(s) I'm deferring-to make visible errors that are important to their stated case; that they have often previously put forward plausible-sounding arguments that were later shown to be wrong; that their credentials are fake; and so on.

Even if you don't have relevant cruxes directly about the topic, put forward cruxes about the people you're deferring-to.

Distinguish your independent components

(I heard this from Andrew Critch or Anna Salamon.)

When you share an opinion, distinguish the part that is originating with you from the part that you are summarizing from other people.

For propositional opinions, this means sharing your first-hand observations separately from your summaries of other people's testimony. E.g.:

• Instead of saying "I think Novavax is better than Moderna", you might say "Gippity says Novavax is better than Moderna, and Googling says it has less side effects ...and I took both and had less side effects from Novavax" or "...and I took both and couldn't tell the difference".
• Instead of saying "I think AI alignment is hard", you might say "All the experts whose writing about AI makes sense to me say that AI alignment is hard, but I haven't tried myself". In this example you have an independent component, which is "which writing about AI makes sense to me". You explain what judgement you are adding in to the mix, and explain what your listener would be trusting if they trust your conclusion, rather than ambiguously posing as an expert.

For values and decisions, this means sharing your desires and your "best guess about what to do, if you were the sole decision-maker", separately from what your actual current plan is, which may be based on your having aggregated the group's values. E.g. you might say "It seems more convenient to go to the DMV and then the grocery store, but I'm not that confident and Alice said the opposite and I'll go along with what she said" rather than "We are going to the grocery store first" which makes it sound like you independently agree that we should go to the grocery store first.

If you're going to update your probability based on others's opinions, then also share your un-updated probability.

This helps avoid information cascades and social miasma.

Retreat to non-deferential cruxes

Are Jews genetically predisposed to be more sneaky than non-Jews? I don't know, probably not, and in order to form an opinion, in practice I would probably have to defer to experts in genetics. But I also don't care very much. Even if we are genetically sneaky, you can't kick us out of the government or ban us from business. The genetics thing isn't a crux, and it shouldn't be for you or for a free society. I don't need to answer that difficult-to-answer question about genetics. If you want to make me even care about genetics in the context of goverment policy, you'd first have to argue the implication from genetics to policy, not anything about genetics itself. I would need to defer about genetics, but I don't need to defer about my judgement that genetics should not effect policy. That's something I can see and argue for myself.

This illustrates a general principle: Often you don't have to make judgements at all. If you can answer the practical, action-affecting questions without fully answering some other question X, then you don't have to form an opinion about X right now.

As another example, I don't know when during development a child gains a soul; but I'm sure they have a soul by age 2 years, and I'm sure they don't have a soul by age 7 days. So I'm confident that it is morally acceptable for parents to choose to destroy 7-day embryos. I would have to defer to neurologists and embryologists about many of the relevant facts for, say, 4-month-old fetuses; but that's not a crux for IVF, and I'm indepedently confident that IVF is morally acceptable.

So, suppose you have a question at hand, and you have some cruxes for that question, and for some of those cruxes you have a non-deferential independent judgement about them. In this case, base your arguments for your position on those cruxes rather than on your deferential judgements. Say "I'm sure a 7-day embryo doesn't have a soul.", not "Experts agree that even a 2-month embryo doesn't have a soul.".

Open problems in graceful deference

• What are more ways to notice when and in what ways you're deferring?

• How are we already deferring, descriptively? Which of these ways are good and bad in what contexts? How can they be generalized, fixed, improved, refined?

• What are some ways to notice that you are deciding or starting to defer? How do you get other people to notice when they are starting to defer?

  • I've done a substantial amount of mentoring for newcomers to the CFAR sphere and AGI existential risk reduction sphere (CFAR, ESPR, MIRI, PIBBSS, SPAR, MATS). I spent a lot of effort trying to get newcomers to confront the pre-paradigm nature of technical AGI alignment, and the strategic uncertainty around AGI X-derisking. So, I've seen a lot of people go around at workshops asking "established experts" such as myself about what's important and what they should be working on. I tried, but never really figured out how, to get them to understand that they were engaging in process of downloading a consensus to defer to, and why it matters that that's what they're doing.

• Intuitive deferential processes.

  • Very often, deference happens unconciously and through forces other than some sensible epistemic updating.
  • What are these other processes? When are they ok and not ok?
  • E.g. what's the deal with naturally subtly deferring to your surrounding social milieu on lots of stuff?
  • E.g. how to interpret when people are deferring to some inexplicit distributed supposed consensus communicated through quasi-linguistic cues?

• How to choose who to defer to? E.g. epistemic spot checks.

• Meta-deference. How should we defer about who to defer to? E.g. who do you trust to tell you who is a reliable expert on something?

• How can awareness of deference be leveraged?

  • Are there more graceful ways to defer that are unlocked by being fully concious that you're deferring from the beginning (e.g. when learning about a new field for the first time)?
  • When you realize that you've been deferring, and you hadn't realized before, what to do? When should you endorse that deference, and how strongly should you endorse it? How quick should you be to stop deferring and instead investigate?
  • What are good and bad ways to orient to others when they are deferring? When they are being deferred-to?

• Compare: deferring to a single person vs. deferring to a group or consensus (e.g. "what virology thinks of COVID") vs. deferring to a multi-party process (e.g. "the jury trial acquitted").

• How to un-defer?

  • How do you prepare to un-defer? See e.g. Planting questions.
  • When to change deference perms for specific deferrees? When and how to fully defer from a deferee, or widen or narrow the scope of deferrence to them?
  • How to prioritize un-deferring? Which questions should you invest in investigating? See e.g. "overhaul key elements ASAP".
  • What other "cleanup" should you do when undeferring? E.g. propagating updates about things you were deferring about; propagating updates about "I shouldn't have been deferring on this question or to this person".

• How to make the deference relationship healthy?

  • See e.g. some of habryka's posts.
  • E.g. how to be a good deferred-to person? E.g. "Do not hand off what you cannot pick up".
  • E.g. how to be a good deferrer, in relation to the deference and the deferree? E.g. "Question the Requirements".

• What to do when you have to defer, but also there's no good deferrees?

  • E.g. what to do when you're in market for lemons? Maybe you can fund people who might themselves become good deferrees.

• How can you recognize when, by deferring, you're feeding yourself to hostile processes? What to do about that?

• How do you gracefully defer as a group?

  • E.g. how do you coordinate to alleviate correlated failures?
  • How do you appropriately aggregate the incentive to investigate independently? Often no one person should unilaterally investigate, if it's just for their own undeferring, but it would be good for a group to have one person investigating so the group can defer less or defer to a more wholesome consensus.

• Third parties.

  • How to notice and understand the deference relationships of other people and groups?
  • How to deal with them? E.g. how to be kind, but also not let people get away with bad behavior because they're just following orders, etc.
  • As a deferree or deferrer, how do you make your deference relationship easier for third parties to interact with suitably?

• Dimensions of deference.

  • Compare: deferring on facts and propositional beliefs; deferring on importance and values; deferring on concepts and questions.
  • There are several reasons that debate or investigation can be infeasible or inappropriate in some contexts. E.g. your stances or beliefs are uncertain, not well-informed, deferential, inexplicit, or weak. How do these relate? E.g. you can have a strong certain inexplicit non-deferential opinion ("I want you to not touch me there; I strongly want that; I'm not uncertain; I can't give a clear explicit explanation of why"). When and how can you and should you untangle deference from other such opaque stances? How to deal with e.g. having a vague blob of cruxes about some question, which is partly deferential and partly your independent intuitions?

• If you're going to be a "foot soldier" for a group or cause, based on deferential stances, how can you alleviate the problems that come from that?

• If you look carefully at my list of Dangers of Deference, you'll see several that aren't adequately addressed by the list of tools in this article. E.g. group effects of meta-deference are mentioned. E.g. the effects of deferring about importance are mentioned; see also "Please don't throw your mind away".

Acknowledgements

Thanks for helpful comments from: Ben Goldhaber, Clara Collier, Linch, Mikhail Samin, Scott Alexander, and Vaniver.