Probability theory is the mathematics governing quantitative degrees of belief.
See Also: Priors, Bayes' Theorem, Mind Projection Fallacy
Although most of the basics and axioms of probability theory are uncontroversial, the interpretations, usages, and relative importance given to each result vary. There are two main interpretations of the concept of probability: the Bayesian (subjectivist, epistemic or evidential) and the frequentist (objectivist) 1. The latter was the major and standard view from late 19th century until the late 20th century, when the Bayesian interpretation gained popularity in many fields of science and philosophy2.
In the Bayesian interpretation, probability is seem as a belief about the credence of an event3 4, whereas frequentist interpretations hold that probability is an objective property of a physical system, a propensity on some accounts5. An event with Bayesian probability of .6 (or 60%) should be interpreted as stating "With confidence 60%, this event contains the true outcome", whereas a frequentist interpretation would view it as stating "Over 100 trials, we should observe event X approximately 60 times." Frequentists tend to base their view of probability in the Law of Large Numbers, holding the expected probability of an individual event to be close to the average of the results obtained from a large number of trials. Thus, only repeatable events can be said to have probabilities.
The Bayesian interpretation, on the other hand, allows one to assign probabilities to both beliefs and events that have never before happened. Bayes' theorem is used to update the probability of those beliefs when presented with new evidence. The Bayesian interpretation aims to model the correct way of thinking rationally when one is forced to deal with ignorance and uncertainty6. Many other fields, such as the study of cognitive biases, use it as the golden reference for rationality.
Many philosophers have argued for a complementary view of probability7 8 9 10, where both interpretations have their places and value. In the paper "A Subjectivist's Guide to Objective Chance", David Lewis has constructed a view where one can incorporate the frequentist interpretation (objective, chance) inside the Bayesian (subjective, credence) as special case. The frequentist probability is a case of a Bayesian probability conditionalized on truth or empirical evidence: "Chance is objectified subjective probability(...). Objectified credence is credence conditional on the truth." 11