This is a post about evaluating possible experts in fields you're not in, where you are somewhat qualified to evaluate their claims.

I try to think along these lines when talking to other engineers about their jobs, but also when figuring out which of two people has got their story straight in online drama.

I frame my model using competence claims, but I think you can use it whenever people are talking about extraordinary events that they witnessed and you haven't, as long as you've got a mental model of what people are likely to lie or exaggerate about.

You can even use my lines of reasoning for legal content once you know what some of the common heterodox legal takes are. (For instance, "fire in a crowded theatre" is not current precedent, but many people act like it is.)

I think my rules of thumb are pretty good, but I really recommend doing diligent research before you boost anything or publicly take a side in anything! When you do that, people other than you have the potential to be hurt or misled.

OK, here's my idea

When someone claims to be fourish standard deviations off the mean -- 160 IQ, for instance -- they're probably lying or mistaken.

Most of the time, when someone says "I'm competent," you have to weigh the odds that they really are against the odds that they're completely wrong. Especially when it's a really tall claim!

Many statements aren't claims of competence per se, but they have strong implications about competence. If you're in France and some guy comes in from England and you ask, "how did you get here?" then four worlds are possible. I'll assign them arbitrary percentage probabilities to make my point:

(9%) They say "I swam," but they didn't
(1%) They say "I swam," and they did
(89%) They say "I didn't swim," and they didn't
(1%) They say "I didn't swim," and they did

Well, based on these numbers and Bayes' Law:

2% of the time, they swam
If they say they swam, then 10% of the time, they swam

(The numbers are arbitrary, but you get the same conclusions as long as you assume that swimming is rare and people rarely lie in a way that implies they're less competent than they actually are.)

So this is all me building up to a general observation, which is that certain claims -- usually claims that sound crazy -- are equally likely to come from a competent or an incompetent person, because they remove the middle.

For instance, in software, you're likel...

Posts

Wikitag Contributions

Comments