To expand on this: Confidence intervals that are accurate for multiple judgements by the same person may be accurate for the same judgement made by multiple people. Normally, we can group everyone's responses and measure how many people were actually right when they said they were 70% sure. This should average out to 70% is because the error is caused by independent variations in each person's estimate. If there's a systematic error, then even if we all accounted for the systematic error in our confidence levels, we would all still fail at the same time if there was an error.

Could you give an example of an experiment that would be too complex for log odds to be useful?

Hello,

I was introduced to Less Wrong by a friend about a year ago. My first impression was of thoughts and opinions that I already had, or had half-thought, but expressed much more clearly. How could I not love it? I eventually read all of the sequences, finding novel but brilliant ideas. I now recommend them to almost everyone I meet. Coincidentally, after I'd started reading the sequences, I found HP:MOR, and had my mind blown when I found out most of them were written by the same person. Currently, I'm trying to read E.T. Jaynes', "Probability Theory: The Logic of Science", but I'm having some trouble, especially since I can't seem to solve any of the examples. If anyone has a solutions guide, or some small hints, I'd greatly appreciate it.