In decision theory, a scoring rule is a measure of performance of probabilistic predictions - made under uncertainty.
As an example of a probabilistic prediction, consider a sports magazine dealing with horse races that gives the winning chance of each horse for each race the day before. If we gather data regarding those predictions and compare it to the actual results, we have a measure – a scoring rule - of the magazine’s performance. This scoring is almost always not linear, however, and there are many different transformations which are widely used.
A proper scoring rule is one that encourages the forecaster to be honest – that is, the expected payoff is maximized by accurately reporting personal beliefs about the predicted event, rather than by gaming the system. These rules include the Logarithmic scoring rule, Spherical scoring rule and Brier/Quadratic scoring rule.
(PDF)