From what I understood on reading the Wikipedia article on Bayesian probability and inferring from how he writes (and correct me if I'm wrong), Eliezer is talking about your "subjective probability." You are a being, have consciousness, and interpret input as information. Given a lot of this information, you've formed an idea that 7 is prime. You've also formed an idea that other people exist, and that the sky is blue, which also have a high subjective probability in your mind because you have a lot of direct information to sustain that ... (read more)
I'm sorry. Eliezer, can you please explain to me what you mean when you say the how certain you are (probability %) that something is true? I've studied a lot of statistics, but I really have no idea what you mean.
If I say that this fair coin in my hand has a 50% chance of coming up heads, then that means that if I flip it a lot of times, then it'll be heads 50% of the times. I can do that with a lot of real, measurable things.
So, what do you mean by, you are 99% certain of something?
"""But if you think you would totally wear that clown suit, then don't be too proud of that either! It just means that you need to make an effort in the opposite direction to avoid dissenting too easily. That's what I have to do, to correct for my own nature."""
I know exactly what you mean. I often see myself dissenting with the majority. Unfortunately, it is difficult to tell if I do so because I am Right, or because I want to be Different.
Sure, I can use logic. But, how do I know I am being a Rationalist, rather than just Rationalizing? It's easy to make up arguments (even coherently logical ones) to support incorrect conclusions. Look at economists.