Being right isn't enough. Confidence is very important.

by MakoYass1 min read7th Apr 20208 comments

12

Rationality
Frontpage

There are two machines that predict weather. One machine predicts the incidence of a storm at 1%, because 1% is the local yearly rate of storms. This machine turns out to be perfectly well calibrated and totally unbiased.

Another machine pays attention to local data, and processes it in a clever way. It always gives numbers like either 98% or 0.05%, depending on the day. Impressively, it also turns out to be perfectly well calibrated and unbiased.

Both are predicting the same category of event, both are perfectly calibrated- the event does happen or fail to happen at the rate they claim it will-, but the confident machine is much more useful than the simpler, low-confidence machine.

Being right and sane means being well calibrated. It imposes no further requirements. Being right and sane is not enough. Moving well in the world requires your predictions to also sometimes be confident.


Feel the pain of confidence's absence. Imagine that an enemy tank has entered your firing range. You can hear it rolling and occasionally clambering through the foliage, but you can't see it, your night cameras are damaged, however, your network holds a lot of good historical data. You figure out the most likely place for it to be given what you can hear and what you all remember, and you fire in that direction. You make the right choice given the data and the cognitive resources you had available to you. You are following a sound algorithm. Nobody can call you insane. Your record as a predictor will have a perfectly triangular histogram.

You miss. Your enemy's night vision is fine, and so they shoot back at you. Some of your parts can be salvaged but they probably wont be. It would have been really good if you'd had more eyes.