Notes on good judgement and how to develop it (80,000 Hours)

by habryka1 min read12th Sep 202016 comments


80,000 HoursRationality

This post by 80,000 hours struck me as more than usually relevant to my interests in developing the art of rationality. It doesn't really say anything new, but it does provide a decent summary of a frame that I think is an important subset of epistemic rationality, in the form of "good judgement". 

More practically, I think of someone with good judgement as someone able to:

  1. Focus on the right questions
  2. When answering those questions, synthesise many forms of weak evidence using good heuristics, and weigh the evidence appropriately
  3. Be resistant to common cognitive biases by having good habits of thinking
  4. Come to well-calibrated conclusions

Owen Cotton-Barratt wrote out his understanding of good judgement, breaking it into ‘understanding’ and ‘heuristics’. His notion is a bit broader than mine.

Here are some closely related concepts:

  • Keith Stanovich’s work on ‘rationality’, which seems to be something like someone’s ability to avoid cognitive biases, and is ~0.7 correlated with intelligence (so, closely related but not exactly the same)
  • The cluster of traits (listed later) that make someone a good ‘superforecaster’ in Philip Tetlock’s work (Tetlock also claims that intelligence is only modestly correlated with being a superforecaster)

Here are some other concepts in the area, but that seem more different:

  • Intelligence: I think of this as more like ‘processing speed’ – your ability to make connections, have insights, and solve well-defined problems. Intelligence is an aid in good judgement – since it lets you make more connections – but the two seem to come apart. We all know people who are incredibly bright but seem to often make dumb decisions. This could be because they’re overconfident or biased, despite being smart.
  • Strategic thinking: Good strategic thinking involves being able to identify top priorities, and develop a good plan for working towards those priorities, and improving the plan over time. Good judgement is a great aid to strategy, but a good strategy can also make judgement less necessary (e.g. by creating a good back-up plan, you can minimise the risks of your judgement being wrong).
  • Expertise: Knowledge of the topic is useful all else equal, but Tetlock’s work (covered more below) shows that many experts don’t have particularly accurate judgement.
  • Decision making: Good decision making depends on all of the above: strategy, intelligence, and judgement.

I do disagree with some of the distinctions being made in the post. As an example, just in the section above, the conception of "Intelligence" as "processing speed" is really flawed, and in-practice intelligence already measures something closer to "good judgement". But overall, the post seems decent as a potential intro into a bunch of rationality stuff.