An average person in the western world probably believes a lot of false things. They probably don't have a great grasp of political economy, or orbital mechanics. How could they, given that they have no way to experience these things. Conversely, they do have a solid grasp that objects fall down, and that fire is hot, because they can experience these things directly. So how on earth do they know (and I do mean know in the philosophical sense) that the earth goes round the sun, or that diseases are caused by tiny creatures too small to see?

The answer is experts. More specifically, an expertise hierarchy.

I

I had a twitter exchange (I won't link it, it's not important, and I can't find it anyway) that went something like this:

Person: David Chalmers is an expert on consciousness [...]

Me: I don't think there are experts on consciousness; I think there are people who have written a lot about it, and that's it

Person: Why? Surely someone who has written about it, and who is well-respected, can be called an expert

Me: [bad explanation]

The fact that it was about consciousness doesn't really matter. The question of whether a field has experts is more about the structure of its knowledge than about the knowledge itself. What I should have said is something like this:

Me: I mean I just can't tell if David Chalmers is correct or not. A bunch of other "experts" disagree with him. They can't all be right. So the method of selecting experts is clearly failing compared to e.g. physics, in which most experts agree.

II

The field with the most expert-y experts is mathematics. The field basically agrees 100% on which theorems are proven given which axioms. This works because it's much easier to check a proof than it is to find a proof.

This creates a kind of hierarchy: it took the mind of Andrew Wiles to prove Fermat's Last Theorem, but many living mathematicians were able to check it. In turn, they had proved their own results, which were checkable and checked by a broader community of mathematicians (either immediately, or when learning the proofs). In my brief years of maths instruction as a wee boy, I was exposed to proofs which I could see to be true; this made it clear to me that those above me in rank were also above me in skill.

This hierarchical stack is locally valid. In the same way that if each step of a proof holds, then the whole proof holds, if I check that my teachers are smarter than me, and they check that their professors are smarter than them, and they check that Terrence Tao is smarter than them, then it holds that Terrence Tao really is the greatest mathematical mind. (And of my classmates, who saw me doing better in maths exams than they did, if they trust me, can trust Terry too.)

III

This holds for a good few other fields as well, but it only holds if every layer is valid. We mostly trust that this doesn't happen in science, but it sometimes does. This might be a single point of active deception: if the Alzheimer's lab at the top is faking results, the everyone is wrong, at least until the fraud is caught (and even then, it might take some time for the corrections to percolate).

It collapses if people are wrong about the general rules of the system—priming research survived for a while because of p-hacking, publication bias—and it comes crashing down if the people involved are doing something other than checking for the truth, like maximizing their social status, or signalling. Insert punching bag social sciences field of your choice here. A chain made of human links is easily broken.

The chain of verification can also be broken by secrecy: even in the west, we have to rely on a lot of trust as to the effectiveness of our military, compared to the effectiveness of our water reservoirs, since there are sharper gaps in the flow of checking others' work (security clearance, official secrets, etc.).

IV

Sometimes, though, the fault really is in our stars, and I think this is where the consciousness example above lies. It's just not that easy to check other people's ideas on consciousness! I've wrestled with this myself in the past. This extends to a lot of philosophy, and it seems to be the problem behind humanity's generally poor philosophical competence.

Humans do well when exposed to tight feedback loops, and very badly when cut off from them. If we can't get feedback from our peers, or even ourselves, as to which of our ideas are good, then we struggle to produce good ideas.

Sometimes it seems like there are multiple competing hierarchies. Different "schools" of thought where the local correction gradients flow towards an attractor, often a single person or a small cluster, whose ideas are irreconcilable with the other attractors. Instead of a single pyramid we have a whole mountain range.

The divergence can be surprisingly early. I can explain Newcomb's Problem to random people at parties (out of mercy, I don't), but I can't (and definitely won't) explain to them why if they choose one option, they should stop voting or recycling, and if they choose the other option, they should be trying to cut deals with aliens in other branches of the multiverse.^[1]

V

What can we do if we find ourselves in such a situation? If we're lucky, we can find some outside-view differentiator between the attractors. This works for, say, economics, where the major schools have been tested in the real world. Unfortunately, philosophy is doubly cursed here! There's no experiment to tell us if Chalmers or Yudkowsky or MacAskill or Aquinas is correct about what makes a being worthy of moral consideration!

In this case, the only thing to do is to be smarter than everyone else, and to take the top-level inside-view arguments on their own terms. That's kind of the whole point of this website we're on, and it's something I value deeply about this community.

If, however, your grand plan requires a bunch of arbitrary people^[2] being on board with you in a non-experty field, you have a much harder task. You don't just have to upskill yourself, you have to upskill those arbitrary people. Better get teaching.

1. ^{^}

   This may be true, it's just quite a reach from "Oh yeah I see, I'll just take the one box then"

2. ^{^}

   I.e. those not selected for falling into your particular attractor in expertise-space