But what exactly constitutes "enough data"? With any finite amount of data, couldn't it be cancelled out if your prior probability is small enough?

Yes, but that's not the way the problem goes. You don't fix your prior in response to the evidence in order to force the conclusion (if you're doing it anything like right). So different people with different priors will have different amounts of evidence required: 1 bit of evidence for every bit of prior odds against, to bring it up to even odds, and then a few more to reach it as a (tentative, as always) conclusion.

Priors Are Useless

by DragonGod 1 min read21st Jun 201722 comments



This post contains Latex. Please install Tex the World for Chromium or other similar Tex typesetting extensions to view this post properly.

Priors are Useless.

Priors are irrelevant. Given two different prior probabilities [;Pr_{i_1};], and [;Pr_{i_2};] for some hypothesis [;H_i;].
Let their respective posterior probabilities be [;Pr_{i_{z1}};] and [;Pr_{i_{z2};].
After sufficient number of experiments, the posterior probability [;Pr_{i_{z1}} \approx [;Pr_{i_{z2};].
Or More formally:
[;\lim_{n \to \infty} \frac{ Pr_{i_{z1}}}{Pr_{i_{z2}}} = 1 ;].
Where [;n;] is the number of experiments.
Therefore, priors are useless.
The above is true, because as we carry out subsequent experiments, the posterior probability [;Pr_{i_{z1_j}};] gets closer and closer to the true probability of the hypothesis [;Pr_i;]. The same holds true for [;Pr_{i_{z2_j}};]. As such, if you have access to a sufficient number of experiments the initial prior hypothesis you assigned the experiment is irrelevant.
To demonstrate.
This is the graph of the above table:
In the example above, the true probability of Hypothesis [;H_i;] [;(P_i);] is [;0.5;] and as we see, after sufficient number of trials, the different [;Pr_{i_{z1_j}};]s get closer to [;0.5;].
To generalize from my above argument:

If you have enough information, your initial beliefs are irrelevant—you will arrive at the same final beliefs.
Because I can’t resist, a corollary to Aumann’s agreement theorem.
Given sufficient information, two rationalists will always arrive at the same final beliefs irrespective of their initial beliefs.

The above can be generalized to what I call the “Universal Agreement Theorem”:

Given sufficient evidence, all rationalists will arrive at the same set of beliefs regarding a phenomenon irrespective of their initial set of beliefs regarding said phenomenon.


Exercise For the Reader

Prove [;\lim_{n \to \infty} \frac{ Pr_{i_{z1}}}{Pr_{i_{z2}}} = 1 ;].