Thanks! Yeah, I still don't get it. I think they'd have been better off doing a classic Implicit Association test.
I thought their method was pretty neat.
Not accounting for the fact that some people may know the correct answer does not amount to "throwing out" their opinions. Quite the opposite -- such people may well be present in the sample, and since they will answer "one" regardless of what stereotypes they hold, the result in the paper may be an underestimate of the extent of distrust-based stereotyping of atheists. In theory, it might get more complicated if different subgroups (e.g. religious vs nonreligious) differ in the fraction of individuals who know the right answer, but this fraction is probably very low across the board.
[EDIT: I think I may have misunderstood what you said in the second paragraph, causing me to overestimate the degree of disagreement here. If this is right, the above might not make much sense to you; in that case just ignore everything except my point that the fraction of people who know the right answer is probably too low to matter.]
I think I may have misunderstood what you said in the second paragraph, causing me to overestimate the degree of disagreement here.
It may be helpful to write explicitly in what sense you have misunderstood. I am now confused about how to understand your comment.
A subsequent reading led to a more charitable interpretation, which prompted me to amend my post to avoid possible confusion (about me sounding like I disagree while making points that HonoreDB would agree with). The paragraph in question (second paragraph in the original post) has since been edited, which might add to your confusion.
A recent set of studies by Gervais, Shariff, and Norenzayan tested whether public dislike of atheists was based more on distrust or revulsion. The first study simply asked directly, and found that American adults report a strong distrust of atheists. However, they worried that explicit answers might be more about signaling, so they did a second study with a more unusual methodology. Questions they asked included (paraphrased) "Is a person who steals money out of a lost wallet more likely to be 1) a teacher or 2) a teacher and an atheist?" and "Is a person who goes all day without noticing he has phlegm on his tie more likely to be 1) a teacher or 2) a teacher and an atheist?" Students, especially religious ones, often answered 2) to the first and 1) to the second, suggesting that it's more about mistrust.
I suspect this isn't actually a more effective way of eliciting stereotypes than asking directly. I think signaling concerns will be just as active in the second study, and there will be a skewing of results in that anyone familiar with the conjunction fallacy (or even the importance of base rates) will answer 1) even if they distrust or are disgusted by atheists. The result will inevitably underestimate dislike of atheists and evince a spurious or exaggerated correlation between such dislike and statistical innumeracy. I think a better way to look for implicit, rather than explicit, stereotyping would be to create an Implicit Association Test. That said, I think the study is still meaningful and I'm intrigued by the methodology.
Thanks to lukeprog, here's the full paper. Relevant excerpt:
Study 1 demonstrated explicit distrust of atheists, but it is possible that, instead of being representative of personal feelings, participants’ explicit responses may have instead reflected cultural norms determining which groups are fair game for criticism and which should be insulated. The varied permissibility of such criticism is itself an interesting indicator of prejudice, but it does not specifically map on to the questions of distrust at the heart of this project. As a result, in Study 2, we adapted a classic conjunction fallacy paradigm (e.g., Tversky & Kahnemann, 1983) to create an indirect measure of distrust for various groups of people.