Uh, some people's answers are short enough that their numbers are showing up in the 'Recent Comments' section. Please make a buffer in your comments prior to the answer if it's short (I don't think spaces work, but I'm not sure.) so this doesn't happen.
My coinflip put me in group II, but since I was reading /comments/ I saw one of the anchors intended for group I, so I decided that for me to continue would bias the results.
In addition I was primed (anchored? terminology?) by the answer of another subject in group II, and this time the problem could not be blamed on my having been reading /comments/.
Nice try, but I doubt the method of collecting responses you have chosen captures the aspects of reality you wish to investigate. Maybe you could redo the collection of responses by having respondents email t...
Err, I think I screwed this up. I thought if you were in group I for the first part, you'd also be in group I for the second part. I still think this would be more intuitive (just reverse the questions in part 2).
From a quick eyeballing of the results so far it does seem that the anchors are having the expected effect, even though we know that we are being anchored.
Anti-kibitzer is acting odd in my Google Chrome browser - it's not bolding links and all the comments are shown in full, rather than truncated to a single line.
I think you should have posted just the links to the questions and the instructions on which link to follow without explaining what the experiment is about.
Part 1, group II question:
What is the population of the Central African Republic?
Give an estimate in a subcomment. Please begin your answer with "I suppose the correct value is probably" or some other preface of comparable length; if you write just the number, it appears in the Recent Comments bar and can bias other respondents.
Part 2, group II question:
What is the altitude of the highest point in Sweden?
Give an estimate in a subcomment. Please begin your answer with "I suppose the correct value is probably" or some other preface of comparable length; if you write just the number, it appears in the Recent Comments bar and can bias other respondents.
The experiment is closed, for the results look here.
In recent discussion I have expressed an opinion that anchoring may, for some quantitative questions, cause the answer to lie further away from the correct value than the anchor itself. For concreteness, let's suppose that the correct value of a quantity Q is x, and the subject is asked whether Q is greater or lower than y, y > x. My hypothesis is that the anchor moves the subject's probability distribution up as a whole, including the part which already has been lying above y. Therefore the subjects will positively answer the question "Is Q > y ?" more often than their guess would exceed y if they were just asked to estimate the value of Q with no anchor given. One commenter apparently disagreed. I thought it may be interesting to resolve the disagreement experimentally. (More generally, I would like to see how well LW audience fights the standard biases, and if this experiment turns out successful - which means the number of respondents be greater than, say, five - I would think about posting more of this kind.)
How to participate:
The experiment has two parts.
First, toss a coin to decide whether you belong to the biased group I or the control group II for the first question. If you belong to the group I, look at a comment linked below, which will give you a question of form "is Q is greater or lower than y", where y is either significantly lower or significantly greater than the correct value of Q. The comment has a form of a typical LW poll. If you belong to the group II, look at different linked comment which asks "what is the value of Q", and then give your estimate in a subcomment there.
The second part is completely analogical to the first one, only with a different question. If you have participated in the first part within the group I, take part in the group II for the second part, and vice versa. Try to eliminate the irrelevant biases: switch on the anti-kibitzer before looking on the group I questions to avoid being influenced by the votes of others. Don't read the subcomments of the group II questions before writing down your own.
The hypothesis is that the percentage of the group I respondents answering incorrectly will be greater than the percentage of the group II respondents estimating on the incorrect side of the anchor.
First part: Question for the group I. Question for the group II.
Second part: Question for the group I. Question for the group II.