This post is an amendment and sort of apology for my LW posts dealing with complexity (1, 2). In these posts I got the terms all wrong. Now I'll try to set it right. Many thanks to taw, JGWeissman and Daniel_Burfoot.
The Solomonoff prior works like this: every bit string is assigned a real number that's equal to the probability of a "random program" outputting that string and then halting, when the "probability" of choosing each program is assumed to be inverse-exponential in its length. (Alternatively, you talk about a "universal prefix Turing machine" that consumes fair coin flips on the input tape.)
In this setup, the words "event", "hypothesis", "probability", "complexity" etc. are all ambiguous. For example, the word "hypothesis" can mean a) an individual program, b) an equivalence class of programs that output the same bit string, or c) a statement about output bit strings, like "the third bit will be 0". The word "event" has exactly the same problem.
Now the trouble is, you can give reasonable-sounding definitions of "probability" and "Kolmogorov complexity" to objects of all three types: (a), (b), and (c). But it's not at all clear what real-world implications these values should have. Does it make sense to talk about prior probability for objects of type (a), given that we can never distinguish two (a)'s that are part of the same (b)? (This is the mistake in saying that MWI has a higher prior probability than collapse interpretations.) Is it useful to talk about K-complexity for objects of type (c), given that a very long statement of type (c) can still have probability close to 1? (This is the mistake in saying Islam must be improbable because the Qur'an is such a thick book.) For that matter, is it at all useful to talk about K-complexity for objects of type (b) which are after all just bit strings, or should we restrict ourselves to talking about their prior probabilities for some reason?
At the moment it seems uncontroversial that we can talk about K-complexity for objects of type (a) - let's call them "programs" - and talk about prior probability for objects of types (b) and (c) - let's call them "sequences of observations" and "statements about the world", respectively. Curiously, this means there's no object for which we have defined both "complexity" and "prior probability", which makes all arguments of the form "complexity is high => probability is low" automatically suspect.
There's another worrying point. Given that "programs" get glued into equivalence classes - "sequences of observations" - anyway, it seems the K-complexity of an individual program is a completely inconsequential variable. You're better off just talking about the length. And the other two kinds of objects don't seem to have useful notions of K-complexity (according to my esteemed commentators who I thanked above), so we can adopt the general rule that mentioning K-complexity in a discussion of physics is always a sign of confusion :-)
What do you say? Is this better? I'm honestly trying to get better!