The experiment is closed, for the results look here.
In recent discussion I have expressed an opinion that anchoring may, for some quantitative questions, cause the answer to lie further away from the correct value than the anchor itself. For concreteness, let's suppose that the correct value of a quantity Q is x, and the subject is asked whether Q is greater or lower than y, y > x. My hypothesis is that the anchor moves the subject's probability distribution up as a whole, including the part which already has been lying above y. Therefore the subjects will positively answer the question "Is Q > y ?" more often than their guess would exceed y if they were just asked to estimate the value of Q with no anchor given. One commenter apparently disagreed. I thought it may be interesting to resolve the disagreement experimentally. (More generally, I would like to see how well LW audience fights the standard biases, and if this experiment turns out successful - which means the number of respondents be greater than, say, five - I would think about posting more of this kind.)
How to participate:
The experiment has two parts.
First, toss a coin to decide whether you belong to the biased group I or the control group II for the first question. If you belong to the group I, look at a comment linked below, which will give you a question of form "is Q is greater or lower than y", where y is either significantly lower or significantly greater than the correct value of Q. The comment has a form of a typical LW poll. If you belong to the group II, look at different linked comment which asks "what is the value of Q", and then give your estimate in a subcomment there.
The second part is completely analogical to the first one, only with a different question. If you have participated in the first part within the group I, take part in the group II for the second part, and vice versa. Try to eliminate the irrelevant biases: switch on the anti-kibitzer before looking on the group I questions to avoid being influenced by the votes of others. Don't read the subcomments of the group II questions before writing down your own.
The hypothesis is that the percentage of the group I respondents answering incorrectly will be greater than the percentage of the group II respondents estimating on the incorrect side of the anchor.