Almost all hypotheses have high complexity. Therefore most high-complexity hypotheses must have low probability.

(To put it differently: let p(n) be the total probability of all hypotheses with complexity n, where I assume we've defined complexity in some way that makes it always a positive integer. Then the sum of the p(n) converges, which implies that the p(n) tend to 0. So for large n the total probability of all hypotheses of complexity n must be small, never mind the probability of any particular one.)

Note: all this tells you only about what happens in the limit. It's all consistent with there being some particular high-complexity hypotheses with high probability.

But why should the probability for lower-complexity hypotheses be any lower?

New Philosophical Work on Solomonoff Induction

by vallinder 1 min read27th Sep 201611 comments


I don't know to what extent MIRI's current research engages with Solomonoff induction, but some of you may find recent work by Tom Sterkenburg to be of interest. Here's the abstract of his paper Solomonoff Prediction and Occam's Razor:

Algorithmic information theory gives an idealised notion of compressibility that is often presented as an objective measure of simplicity. It is suggested at times that Solomonoff prediction, or algorithmic information theory in a predictive setting, can deliver an argument to justify Occam's razor. This article explicates the relevant argument and, by converting it into a Bayesian framework, reveals why it has no such justificatory force. The supposed simplicity concept is better perceived as a specific inductive assumption, the assumption of effectiveness. It is this assumption that is the characterising element of Solomonoff prediction and wherein its philosophical interest lies.