"I'd be willing to bet $1,000 with anyone that the eventual total error of my forecasts will be less than the 65th percentile of my specified predicted error."

I think this is equivalent to applying a non-linear transformation to your proper scoring rule. When things settle, you get paid S(p) both based on the outcome of your object-level prediction p, and your meta prediction q(S(p)).



where B is the "betting scoring function".

This means getting the scoring rules to work while preserving properness will be tricky (though not necessarily impossible).

One mechanism that might help is that if each player makes one object prediction p and one meta prediction q, but for resolution you randomly sample one and only one of the two to actually pay out.

ozziegooen's Shortform

by ozziegooen 31st Aug 2019127 comments