You can train experts to be calibrated in different ways. If you train experts to be calibrated to pick the right probability on GPOpen where probability is done in steps on 1, I don't think those experts will be automatically calibrated to distinguish a p=0.00004 event from a p=0.00008.

Experts would actually need to be calibrated on getting probabilities inside the tail right. I don't think we know how to do calibration training for that tail.

ozziegooen's Shortform

by ozziegooen 31st Aug 2019127 comments