LESSWRONGTags
LW

Probability Theory

''' Probability theory ''' is a field of mathematics whichthat studies random variables and processes.

See Also: Priors, Bayes' Theorem, Mind Projection Fallacy

Bayesian vs Frequentist Interpretations of Probability

Blog postsNotable Posts

See also

  1. It’s worth mentioning that within the Bayesian interpretation there are also viewpoints referred to as "objectivist" and "subjectivist". See Probability is Subjectively Objective for more on this.↩
  2. WOLPERT, R.L. (2004) “A conversation with James O. Berger”, Statistical science, 9, 205–218.↩
  3. BERNARDO, J. M. & SMITH, A. F. M. (1994) “Bayesian Theory”. Wiley.↩
  4. JAYNES, E. T. (1996) ”Probability theory: The logic of science.'' Available from http://bayes.wustl.edu/etj/prob.html↩
  5. POPPER, Karl.(1959) "The propensity interpretation of probability" The British Journal of the Philosophy of Science, Vol. 10, No. 37. (May, 1959), pp.25-42. Available at: http://www.hum.utah.edu/~mhaber/Documents/Course%20Readings/Popper_Propensity_BJPS1959LITE.pdf↩
  6. KORB, Kevin & NICHOLSON, Ann. (2010) "Bayesian Artificial Intelligence". CRC Press. p. 28↩
  7. CARNAP, Rudolf. (1945). "The Two Concepts of Probability", Philosophy and Phenomenological Research 5, 513-532.↩
  8. JEFFREY, Richard C. (1965). "The Logic of Decision." New York: McGraw-Hill.↩
  9. LEWIS, David. (1980). “A Subjectivist's Guide to Objective Chance”. In JEFFREY, Richard C. (ed.), Studies in Inductive Logic and Probability. University of California Press. Available at: http://fitelson.org/probability/Lewis_asgtoc.pdf↩
  10. COX, R. T. (1946) “Probability, frequency and reasonable expectation,'' American Journal of Physics, vol. 14, no. 1, pp. 1-13↩
•
Created by joaolkf at 4y
  1. It’s worth mentioning that insidewithin the Bayesian interpretation,interpretation there are also views called objectivists.viewpoints referred to as "objectivist" and "subjectivist". See Probability is Subjectively Objective for more on this.↩
  2. WOLPERT, R.L. (2004) “A conversation with James O. Berger”, Statistical science, 9, 205–218.↩
  3. BERNARDO, J. M. & SMITH, A. F. M. (1994) “Bayesian Theory”. Wiley.↩
  4. JAYNES, E. T. (1996) ”Probability theory: The logic of science.'' Available from http://bayes.wustl.edu/etj/prob.html↩
  5. POPPER, Karl.(1959) "The propensity interpretation of probability" The British Journal of the Philosophy of Science, Vol. 10, No. 37. (May, 1959), pp.25-42. Available at: http://www.hum.utah.edu/~mhaber/Documents/Course%20Readings/Popper_Propensity_BJPS1959LITE.pdf↩
  6. KORB, Kevin & NICHOLSON, Ann. (2010) "Bayesian Artificial Intelligence". CRC Press. p. 28↩
  7. CARNAP, Rudolf. (1945). "The Two Concepts of Probability", Philosophy and Phenomenological Research 5, 513-532.↩
  8. JEFFREY, Richard C. (1965). "The Logic of Decision." New York: McGraw-Hill.↩
  9. LEWIS, David. (1980). “A Subjectivist's Guide to Objective Chance”. In JEFFREY, Richard C. (ed.), Studies in Inductive Logic and Probability. University of California Press. Available at: http://fitelson.org/probability/Lewis_asgtoc.pdf↩
  10. COX, R. T. (1946) “Probability, frequency and reasonable expectation,'' American Journal of Physics, vol. 14, no. 1, pp. 1-13↩

''' Probability theory ''' is a field of mathematics, itmathematics which studies random variables and processes. The evolution of these processes and variables over time take many possible paths departing from the same initial condition and doesn't follow deterministic rules. Such evolution can be mathematically represented in the form of a probability distribution.

The basic axioms of probability theory can be intuitively stated as:

  • The sum of the probabilities of all elementary events is 1.
  • For any pair of arbitrary events, the joint probability of both is given by the sum of the probabilities of the elementary events which are subsets of both of them at the same time.
  • For any pair of arbitrary events, the disjoint probability of both is given by the sum of the probabilities of the elementary events which are subsets of at least one of them.

Although most of the basics and axioms of probability theory are uncontroversial, the interpretations, usages, and relative importance given to each result vary. There are two main interpretations of the concept of probability: the Bayesian (subjectivist, epistemic or evidential) and the frequentist (objectivist) 1. The latter was the major and standard view from late 19th century until the late 20th century, when the Bayesian interpretation gained popularity in many fields of science and philosophy2.Bayesian networks show themselves to be extremely fruitful in the field of Artificial Intelligence.

The Bayesian interpretation of probability theory aims to model the correct way rational thinking ought to be when bounded by ignorance and uncertainty 3. It's considered by many others fields as providing the golden reference for rationality – i.e.: in the study of cognitive biases. Its main theorem, Bayes' theorem, follows directly from the basic Probability Axioms, and describe the basics relations between dependent probabilities. Bayesians see this result as prescribing a method on how to update one’s probabilities given evidence, and this update as a defining aspect of the concept of probability. Whereas frequenstists tend to based their view of probability in the Law of Large Numbers, holding the expected probability of an individual event must be close to the average of the results obtained from a large number of trials.

In the Bayesian interpretation, probability is seem as a belief about the credence of an event43 54, whereas frequentist interpretations hold that probability is an objective property of a physical system, a propensity on some accounts65. An event with Bayesian probability of .6 (or 60%) should be interpreted as stating "With confidence 60%, this event contains the true outcome", whereas a frequentist interpretation would view it as stating "Over 100 trials, we should observe event X approximately 60 times." Frequentists tend to base their view of probability in the Law of Large Numbers, holding the expected probability...

Read More (360 more words)

In the Bayesian interpretation, probability is seem as a belief about the credence of an event4 5, whereas frequentist interpretations hold that probability is an objective property of a physical system, a propensity on some accounts6. An event with Bayesian probability of .6 (or 60%) should be interpreted as stating "With confidence 60%, this event contains the true outcome", whereas a frequentist interpretation would view it as stating "Over 100 trials, we should observe event X approximately 60 times." The Bayesian interpretation holds one should recursively appliedapply the theorem in order to update his beliefs, when presented with new evidence the previous estimation becomes a prior to the next. It’s possible to assign probabilities to ideas or prepositions which aren’t empirical. Given some prior assumptions, in some cases the Bayesian probability can be made to be equal with the empirical frequencies7 8.

Many philosophers have argued for a complementary view of probability9 10 11 12, where both interpretations have their places and value. In the paper "A Subjectivist's Guide to Objective Chance", David Lewis has constructed a view where one can incorporate the frequentist interpretation (objective, chance) inside the Bayesian (subjective, credence) as special case. The frequentist probability is a case of a Bayesian probability conditionalized on truth or empirical evidence: "Chance is objectified subjective probability(...). Objectified credence is credence conditional on the truth." 13

Many philosophers have argued for a complementary view of probability9 10 11 12, where both interpretations have their places and value. In "A Subjectivist's Guide to Objective Chance", David Lewis has constructed a view where one can incorporate the frequentist interpretation (objective, chance) inside the Bayesian (subjective, credence) as special case. The frequentist probability is a case of a Bayesian probability conditionalized on truth or empirical evidence: "Chance is objectified subjective probability (.probability(...). Objectified credence is credence conditional on the truth." 13

Many philosophers have argued for a complementary view of probability9 10 11 12, where both interpretations have their places and value. David Lewis has constructed a view where one can incorporate the frequentist interpretation (objective, chance) inside the Bayesian (subjective, credence) as special case. The frequentist probability is a case of a Bayesian probability conditionalized on truth or empirical evidence: "Chance is objectified subjective probability (...). Objectified credence is credence conditional on the truth." 13

In the Bayesian interpretation, probability is seem as a belief held by someone about the credence of an event4 5, whereas frequentist interpretations hold that probability is an objective property of a physical system, a propensity on some accounts6. An event with Bayesian probability of .6 (or 60%) should be interpreted as stating "With confidence 60%, this event contains the true outcome", whereas a frequentist interpretation would view it as stating "Over 100 trials, we should observe event X approximately 60 times." The Bayesian interpretation holds one should recursively applied the theorem in order to update his beliefs, when presented with new evidence the previous estimation becomes a prior to the next. It’s possible to assign probabilities to ideas or prepositions which aren’t empirical. Given some prior assumptions, in some cases the Bayesian probability can be made to be equal with the empirical frequencies7 8.

  1. It’s worth mentioning that inside the Bayesian interpretation, there are also views called objectivists.↩
  2. WOLPERT, R.L. (2004) “A conversation with James O. Berger”, Statistical science, 9, 205–218.↩
  3. KORB, Kevin & NICHOLSON, Ann. (2010) "Bayesian Artificial Intelligence". CRC Press. p. 28↩
  4. BERNARDO, J. M. & SMITH, A. F. M. (1994) “Bayesian Theory”. Wiley.↩
  5. JAYNES, E. T. (1996) ”Probability theory: The logic of science.'' Available from http://bayes.wustl.edu/etj/prob.html↩
  6. POPPER, Karl.(1959) "The propensity interpretation of probability" The British Journal of the Philosophy of Science, Vol. 10, No. 37. (May, 1959), pp.25-42. Available at: http://www.hum.utah.edu/~mhaber/Documents/Course%20Readings/Popper_Propensity_BJPS1959LITE.pdf↩
  7. COX, R. T. (1946) “Probability, frequency and reasonable expectation,'' American Journal of Physics, vol. 14, no. 1, pp. 1-13↩
  9. CARNAP, Rudolf. (1945). "The Two Concepts of Probability", Philosophy and Phenomenological Research 5, 513-532.↩
  10. JEFFREY, Richard C. (1965). "The Logic of Decision." New York: McGraw-Hill.↩
  11. LEWIS, David. (1980). “A Subjectivist's Guide to Objective Chance”. In JEFFREY, Richard C. (ed.), Studies in Inductive Logic and Probability. University of California Press. Available at: http://fitelson.org/probability/Lewis_asgtoc.pdf↩
  1. It’s worth mentioning that inside the Bayesian interpretation, there also views called objectivists.↩
  2. WOLPERT, R.L. (2004) “A conversation with James O. Berger”, Statistical science, 9, 205–218.↩
  3. KORB, Kevin & NICHOLSON, Ann. (2010) "Bayesian Artificial Intelligence". CRC Press. p. 28↩
  4. BERNARDO, J. M. & SMITH, A. F. M. (1994) “Bayesian Theory”. Wiley.↩
  5. JAYNES, E. T. (1996) ”Probability theory: The logic of science.'' Available from http://bayes.wustl.edu/etj/prob.html↩
  6. POPPER, Karl.(1959) "The propensity interpretation of probability" The British Journal of the Philosophy of Science, Vol. 10, No. 37. (May, 1959), pp.25-42. Available at: http://www.hum.utah.edu/~mhaber/Documents/Course%20Readings/Popper_Propensity_BJPS1959LITE.pdf↩
  7. COX, R. T. (1946) “Probability, frequency and reasonable expectation,'' American Journal of Physics, vol. 14, no. 1, pp. 1-13↩
  9. CARNAP, Rudolf. (1945). "The Two Concepts of Probability", Philosophy and Phenomenological Research 5, 513-532.↩
  10. JEFFREY, Richard C. (1965). "The Logic of Decision." New York: McGraw-Hill.↩
  11. LEWIS, David. (1980). “A Subjectivist's Guide to Objective Chance”. In JEFFREY, Richard C. (ed.), Studies in Inductive Logic and Probability. University of California Press. Available at: http://fitelson.org/probability/Lewis_asgtoc.pdf↩

''' Probability theory ''' is a field of mathematics, it studies random variables and processes. The evolution of these processes and variables over time take many possible paths departing from the same initial condition and doesn't follow deterministic rules. Such evolution can be mathematically represented in the form of a probability distribution.

The basic axioms of probability theory can be intuitively stated as:

  • The sum of the probabilities of all elementary events is 1.
  • For any pair of arbitrary events, the joint probability of both is given by the sum of the probabilities of the elementary events which are subsets of both of them at the same time.
  • For any pair of arbitrary events, the disjoint probability of both is given by the sum of the probabilities of the elementary events which are subsets of at least one of them.

Although most of the basics and axioms of probability theory are uncontroversial, the interpretations, usages, and relative importance given to each result vary. There are two main interpretations of the concept of probability: the Bayesian (subjectivist, epistemic or evidential) and the frequentist (objectivist) 1. The latter was the major and standard view from late 19th century until the late 20th century, when the Bayesian interpretation gained popularity in many fields of science and philosophy2. Bayesian networks show themselves to be extremely fruitful in the field of Artificial Intelligence.

The Bayesian interpretation of probability theory aims to model the correct way rational thinking ought to be when bounded by ignorance and uncertainty 3. It's considered by many others fields as providing the golden reference for rationality – i.e.: in the study of cognitive biases. Its main theorem, Bayes' theorem, follows directly from the basic Probability Axioms, and describe the basics relations between dependent probabilities. Bayesians see this result as prescribing a method on how to update one’s probabilities given evidence, and this update as a defining aspect of the concept of probability. Whereas frequenstists tend to based their view of probability in the Law of Large Numbers, holding the expected probability of an individual event must be close to the average of the results obtained from a large number of trials.

In the Bayesian interpretation, probability is seem as a belief held by someone 45, whereas frequentist interpretations hold that probability is an objective property of a physical system, a propensity on some accounts6. An event with Bayesian probability of .6 (or 60%) should be interpreted as stating "With confidence 60%, this event contains the true outcome", whereas a frequentist interpretation would view it as stating "Over 100 trials, we should observe event X approximately 60 times." The Bayesian interpretation holds one should recursively applied the theorem in order to update his beliefs, when presented with new evidence...

Read More (219 more words)