Uh, some people's answers are short enough that their numbers are showing up in the 'Recent Comments' section. Please make a buffer in your comments prior to the answer if it's short (I don't think spaces work, but I'm not sure.) so this doesn't happen.
My coinflip put me in group II, but since I was reading /comments/ I saw one of the anchors intended for group I, so I decided that for me to continue would bias the results.
In addition I was primed (anchored? terminology?) by the answer of another subject in group II, and this time the problem could not be blamed on my having been reading /comments/.
Nice try, but I doubt the method of collecting responses you have chosen captures the aspects of reality you wish to investigate. Maybe you could redo the collection of responses by having respondents email t...
Err, I think I screwed this up. I thought if you were in group I for the first part, you'd also be in group I for the second part. I still think this would be more intuitive (just reverse the questions in part 2).
From a quick eyeballing of the results so far it does seem that the anchors are having the expected effect, even though we know that we are being anchored.
Anti-kibitzer is acting odd in my Google Chrome browser - it's not bolding links and all the comments are shown in full, rather than truncated to a single line.
I think you should have posted just the links to the questions and the instructions on which link to follow without explaining what the experiment is about.
Part 1, group II question:
What is the population of the Central African Republic?
Give an estimate in a subcomment. Please begin your answer with "I suppose the correct value is probably" or some other preface of comparable length; if you write just the number, it appears in the Recent Comments bar and can bias other respondents.
Part 2, group II question:
What is the altitude of the highest point in Sweden?
Give an estimate in a subcomment. Please begin your answer with "I suppose the correct value is probably" or some other preface of comparable length; if you write just the number, it appears in the Recent Comments bar and can bias other respondents.
I note that I don't know what the Central African Republic is, but I'm guessing that it's somewhere in Africa. I'm probably best off estimating that it has the average population of a state in Africa, but I don't know what this is either. It will be much smaller than the US population (P>.999), but that doesn't help me....much. I note that this fact in itself implies that I can form a probability distribution. My estimated probability of the Central African Republic being smaller than the US is .999, so I have .999 to distribute within the range 0>450,000,000 (4.5x10^8 is my estimate of the US population).
I don't think that I should use a hypothesis of complete ignorance here because I think that I would have a greater chance of hearing of it the more people it had, but this isn't very dependable considering that I don't follow politics that much. I also note that you may have made the name up, giving it a population of zero. But wouldn't that render the experiment invalid? I can't immediately see why. You're also posting this on April Fools, of all days, and we haven't had a joke yet, so I weight this option more than the others.
Yet we're trying to minimize error here, not pick the integer which we believe has the highest probability of being a correct answer. To take this possibility into account, I will multiply my guess of an average nation's population by .4.
What data can I use to make that guess. Well, I was at a Model United Nations conference once, and I thought that there were about 94 nations present. I suppose that there are about 40 nations not present in the UN, so let's run 6.5 billion/136. We get 46 million. I will multiply that number by .4 to get 18.7 million.
Reposted: put this in the wrong place the first time.
Upvoted for being the sort of analysis one should do to avoid anchoring. Ironically it ended up much closer to the anchor than to the correct value.
The experiment is closed, for the results look here.
In recent discussion I have expressed an opinion that anchoring may, for some quantitative questions, cause the answer to lie further away from the correct value than the anchor itself. For concreteness, let's suppose that the correct value of a quantity Q is x, and the subject is asked whether Q is greater or lower than y, y > x. My hypothesis is that the anchor moves the subject's probability distribution up as a whole, including the part which already has been lying above y. Therefore the subjects will positively answer the question "Is Q > y ?" more often than their guess would exceed y if they were just asked to estimate the value of Q with no anchor given. One commenter apparently disagreed. I thought it may be interesting to resolve the disagreement experimentally. (More generally, I would like to see how well LW audience fights the standard biases, and if this experiment turns out successful - which means the number of respondents be greater than, say, five - I would think about posting more of this kind.)
How to participate:
The experiment has two parts.
First, toss a coin to decide whether you belong to the biased group I or the control group II for the first question. If you belong to the group I, look at a comment linked below, which will give you a question of form "is Q is greater or lower than y", where y is either significantly lower or significantly greater than the correct value of Q. The comment has a form of a typical LW poll. If you belong to the group II, look at different linked comment which asks "what is the value of Q", and then give your estimate in a subcomment there.
The second part is completely analogical to the first one, only with a different question. If you have participated in the first part within the group I, take part in the group II for the second part, and vice versa. Try to eliminate the irrelevant biases: switch on the anti-kibitzer before looking on the group I questions to avoid being influenced by the votes of others. Don't read the subcomments of the group II questions before writing down your own.
The hypothesis is that the percentage of the group I respondents answering incorrectly will be greater than the percentage of the group II respondents estimating on the incorrect side of the anchor.
First part: Question for the group I. Question for the group II.
Second part: Question for the group I. Question for the group II.