I notice that expert opinion always tends to get the short shrift in these. But for many fields I find myself hoping for a competent expert who had gone through all the evidence and could tell me which of the meta-analyses and RCTs were actually reliable, which hypotheses are basically zombie theories, et cetera.

Reply

[-]B Jacobs5y20

And by what metric do you separate the competent experts from the non-competent experts? I also prefer listening to experts because they can explain vast amounts of things in "human" terms, inform me how different things interact and subsequently answer my specific questions. It's just that for any single piece of information you'd rather have a meta-analysis backing you up than an expert opinion.

Reply

[-]Kaj_Sotala5y40

And by what metric do you separate the competent experts from the non-competent experts?

There's no hard-and-fast rule, obviously, just as there's no hard-and-fast rule for figuring out which meta-analyses you can trust (for problems with meta-analyses, see e.g. [1, 2, 3, 4]). But if the experts explicitly discuss the reasons behind their opinions and e.g. why they think that one particular meta-analysis is decent but another one is flawed, you can try try to evaluate how reasonable their claims sound.

LESSWRONG
LW

LESSWRONG
LW

7

Hierarchy of Evidence

7

7