When I started doing epistemic spot checks, I would pick focal claims and work to verify them. That meant finding other sources and skimming them as quickly as possible to get their judgement on the particular claim. This was not great for my overall learning, but it’s not even really good for claim evaluation: it flattens complexity and focuses me on claims with obvious binary answers that can be evaluated without context. It also privileges the hypothesis by focusing on “is this claim right?” rather than “what is the truth?”.

So I moved towards reading all of my sources deeply, even if my selection was inspired by a particular book’s particular claim. But this has its own problems.

In both The Oxford Handbook of Children and Childhood Education in the Ancient World and Children and Childhood in Roman Italy, my notes sometimes degenerate into “and then a bunch of specifics”. “Specifics” might mean a bunch of individual art pieces, or a list of books that subtly changed a field’s framing.  This happens because I’m not sure what’s important and get overwhelmed.

Knowledge of importance comes from having a model I’m trying to test. The model can be external to the focal book (either from me, or another book), or from it. E.g. I didn’t have a a particular frame on the evolution of states before starting Against the Grain, but James C. Scott is very clear on what he believes, so I can assess how relevant various facts he presents are to evaluating that claim.

[I’m not perfect at this- e.g., in The Unbound Prometheus, the author claims that Europeans were more rational than Asians, and that their lower birth rate was evidence of this. I went along with that at the time because of the frame I was in, but looking back, I think that even assuming Europe did have a lower birth rate, it wouldn’t have proved Europeans were more rational or scientifically minded. This is a post in itself.]

If I’d come into The Oxford Handbook of Children and Childhood Education in the Ancient World or Children and Childhood in Roman Italy with a hypothesis to test, it would have been obvious information was relevant and what wasn’t. But I didn’t, so it wasn’t, and that was very tiring.

The obvious answer is “just write down everything”, and I think that would work with certain books. In particular, it would work with books that could be rewritten in Workflowy: those with crisp points that can be encapsulated in a sentence or two and stored linearly or hierarchically. There’s a particular thing both books did that necessitated copying entire paragraphs because I couldn’t break it down into individual points.

Here’s an example from Oxford Handbook…

“Pietas was the term that encompassed the dutiful respect shown by the Romans towards their gods, the state, and members of their family (Cicero Nat. Deor. 1.116; Rep. 6.16; O . 2.46; Saller 1991: 146–51; 1998). is was a concept that children would have been socialized to understand and respect from a young age. Between parent and child pietas functioned as a form of reciprocal dutiful affection (Saller 1994: 102–53; Bradley 2000: 297–8; Evans Grubbs 2011), and this combination of “duty” and “affection” helps us to understand how the Roman elite viewed and expressed their relationship with their children.”

And from Children and Childhood…

“No doubt families often welcomed new babies and cherished their children, but Roman society was still struggling to establish itself even in the second century and many military, political, and economic problems preoccupied the thoughts and activities of adult Romans”

I summarized that second one as “Families were distracted by war and such up through 0000 BC”, which is losing a lot of nuance. It’s not impossible to break these paragraphs down into constituent thoughts, but it’s ugly and messy and would involve a lot of repetition. The first mixing up what pietas is with how and who it was expressed to. The second is combining a claim about the state of Rome with the state’s effects.

This reveals that calling the two books “lists of facts” was incomplete. Lists of facts would be easier to take notes on.  These authors clearly have some concepts they are trying to convey, but because they’re not cleanly encapsulated in the author’s own mind it’s hard for me to encapsulate them. It’s like trying to lay the threads of a gordian knot in an organized fashion.

So we have two problems: books which have laid out all their facts in a row but not connected them, and books which have entwined their facts too roughly for them to be disentangled. These feel very similar to me but when I write it out the descriptions sure sound like two completely different problems.

Let me know how much sense this makes, I can’t tell if I’ve written something terribly unpolished-but-deep or screamingly shallow.

New to LessWrong?

New Comment
5 comments, sorted by Click to highlight new comments since: Today at 11:44 AM
So we have two problems: books which have laid out all their facts in a row but not connected them, and books which have entwined their facts too roughly for them to be disentangled. These feel very similar to me but when I write it out the descriptions sure sound like two completely different problems.

Possible unifying viewpoint: the problem is a lack of gears.

Two key properties of gears:

  • they give us a factorization of the system: each gear' behavior can be viewed in isolation (i.e. locally)
  • they tell us how the gears connect: the whole (i.e. global) system's behavior is defined by the (local) behavior of individual gears, plus their connection structure

Disconnected facts fail the latter; "facts too roughly entangled" fail the former.

This is on to something. I would like each bullet point in my notes to be one gear, and I feel unhappy when it's either a mishmash of several undifferentiated gears, or when the gears don't connect to anything and thus are their own kind of undifferentiated.

So I moved towards reading all of my sources deeply, even if my selection was inspired by a particular book’s particular claim.

The evolution of the process makes sense, but I'm curious if it still makes much sense to call it an "epistemic spot check" if that's the case?

Not really, but what it's going to become isn't clear yet and I want to limit the number of name changes.