Prestigious scientists can play this role even with anonymous review. They can publicize their support of a new and radical idea through talks at conferences, word of mouth, and other informal channels (like blogging). This will increase attention to their new idea, especially since many scientists like to keep track of what the top researchers in their field are doing. And if other scientists give extra weight to the views of their high status colleagues, they will also raise their probability estimates for the hypothesis. So the new idea can receive a boost before any paper is published. Then when papers supporting the radical new hypothesis are submitted for anonymous review, they will have a better chance of being published whether or not the prestigious scientist is an author.
This is an interesting and important point - the sort of thing where once you hear it, you realize all the evidence pointing in that direction that your mind just ignored because of the cached proverb "old scientists resist progress".
I wonder if there's some selection bias inherent in the studies presented here. Assuming that it has been established that older scientists are more willing to accept new controversial hypotheses than younger scientists, has it also been established that they differentially accept good new controversial hypotheses? What I see here is that they tended to embrace the big paradigm shifts relatively early, but it doesn't say anything about older scientists' tendencies to embrace controversial hypotheses that ended up later being discredited. Specifically, Linus Pauling's obsession with Vitamin C megadosing later in life springs to mind.
Reviewers are likely to have a hard time shooting down the work of anyone they know personally, and in specialized sciences, the probability that a paper was written by a friend or former colleague of the reviewer is high enough to be a problem. On the other hand, credibility does matter; if an unestablished author observes something strange, it's likely that he's made a mistake, but an old hand making the same observation should not be ignored. Perhaps instead of a name, reviewers should be given an abstract indication of the author's credibility, such as the author's faculty title, or the number of times they've published before.
Reviewers are likely to have a hard time shooting down the work of anyone they know personally, and in specialized sciences, the probability that a paper was written by a friend or former colleague of the reviewer is high enough to be a problem.
On the other hand, people have different, and often recognizable writing styles. If you're in a specialized science with a small enough circle, then you may be able to recognize a paper written by your friend (or your foe!) even though it was anonymized.
Or maybe you may even know that your friend (or foe?) is working on a paper on a specific topic just from reading their blog/Facebook/Twitter/etc.
I'd be interested in how exactly something like that would work, let's be careful not to make the object of science publishing papers.
I'm not sure how helpful this sort of meta level second guessing of the minds of others is. Reviewers should do their best to review the merits of the paper, regardless of what they know or guess about author. Any epistemic benefit you could derive from considering the author's identity is washed out by the huge risk that you're simply using that as an excuse to make the conclusions you wanted to make in the first place.
This is why we disdain ad hominem arguments. Not because they are invalid per se, but because we know that all too often they only serve as an excuse not to think, not to consider all the evidence.
On the other hand, maybe if papers weren't anonymous, those with prestige could easily be given too much deference. How much Protection From Editors do they actually deserve?
Exactly. It seems unlikely that prestigious researchers will be unable to publish their brilliant but unconventional idea because they can't fully utilize their fame to sway editors. In fact, prestigious researchers have exactly what is needed to ensure their idea will take hold if it has merit: job security. They have plenty of time to nurture and develop their idea until it is accepted.
The title of the post is "Does blind review slow down science?", not "Does blind review stop science?". The prestigious researchers may have the time, but there are plenty of members of humanity that don't. Science is slow enough as it is. We would be well advised to consider any factors that may speed up progress.
There seems to be to assumptions that need to be correct for blind review to be detrimental: Both A) Older established scientists are more likely to be correct when they are positing an anti-establishment thinking explanantion than a younger, less established scientist, and B) those scientists are nonetheless no more capable of marshaling the required set of arguments to do so when faced with blind review than that younger scientist.
I have no issue with A), but B) seems to me to be supremely unlikely - the very factors of an established pattern of rigor that make it more likely that an older scientist may be a safer bet when he breaks from the establishment than I am, also would appear to make it more likely that he or she can establish the case without relying upon reputation.
I might be wrong, but I wouldn't have to fake surprise at learning I was.
I think your argument, on the other hand, relies on the assumption that the reviewers are objective and rational. In an ideal world, they would be. In practice, however, just seeing a conclusion they don't agree with may be enough that they won't evaluate the argument wholely on its merits.
As a somewhat over-the-top example: suppose you were a reviewer in a popular biology journal which, due to its popularity, also got lots of low-quality submissions. Now you are told to review an article whose author you don't know, but which argues that our common conception of how evolution works is actually really flawed. You know that lots of deluded creationists bombard the journal with submissions arguing points like these, and you have had the misfortune of personally reviewing several before this. (They were all completely worthless.)
Then an alternate universe where the situation is exactly the same, but the journal hasn't implemented a policy for double-blind reviews. You are told the name of the author, which is Richard Dawkins.
In which scenario do you think you'll give the submission a more fair hearing?
First I should state that I disagree with anonymous review for the same reasons that I disagree with an unaccountable judiciary - the negative effects on responsibility.
However, there are several problems with the theory in this essay - the most important being that the editors know who the writer or researcher is and can decide to go ahead and publish on that score no matter what the reviewers say. The editors have a strong incentive to advance novel but true theories in that it will advance the reputation of the journal.
If reviewers anonymity is abolished, how many would dare to submit a negative review, knowing that eventually their own work might be refereed by the author?
This is a bit off-topic so I'm not going into any detail here, but you might check out this book by Max Boot, "Out of Order: Arrogance, Corruption, & Incompetence on the Bench", a large proportion of the problems he wrote about arose from judges not being personally responible for their actions on the bench.
Also, more generally, I am a libertarian largely because I believe that everyone is totally and completely responsible for their own actions. Even if someone is holding a gun to your head, you decide what you do in response (and are responsible for letting yourself get in that position). Or if you are drunk or drugged, you are responsible for putting yourself in that position and therefore for what you do while that way.
I mean that people should bear some part of the forseeable costs of their actions. I say "some part" because the actions of others also influence costs, and stress "foreseeable" because in any complex system things interact to such an extent that only very direct results can actually be attributed reliably to any one party. Most attributions of "fault" in complex systems is scapegoating or motivated by interpersonal status games.
I don't understand your objection to anonymous review on the basis of accountability. Doesn't "anonymous review" in this context just mean that the reviewers don't know the authors and affiliations of the papers they're reviewing? In that case, what is there to be accountable for? The reviewers themselves aren't any more anonymous in "anonymous review" than in standard review, are they?
Doesn't "anonymous review" in this context just mean that the reviewers don't know the authors and affiliations of the papers they're reviewing?
In this context, yes, that's the only thing it means.
Maybe I was wrong about that, but I also understood it to mean that the reviewer was also unknown to the author, even after the review. I have heard several stories (can't remember the sources; possibly only urban-scientific legends) of reviewers giving poor reviews of work that could have pre-empted things they were currently working on. And similar self-serving tactics.
Previously, Robin Hanson pointed out that even if implementing anonymous peer review has an effect on the acceptance rate of different papers, this doesn't necessarily tell us the previous practice was biased. Yesterday, I ran across an interesting passage suggesting one way that anonymous review might actually be harmful:
Second, fame may confer license. If a person has done valuable work in the past, this increases the probability that his current work is also valuable and induces the audience to suspend its disbelief. He can therefore afford to thumb his nose at the crowd. This is merely the obverse of the "shamelessness" of the old, which Aristotle discussed. Peter Messeri argues in this vein that "senior scientists are better situated than younger scientists to withstand adverse consequences of public advocacy of unpopular positions," and that this factor may explain why the tendency for older scientists to resist new theories is, in fact, weak. And remember Kenneth Dover's negative verdict on old age (chapter 5)? He offered one qualification: "There just aren't any [aspects of old age which compensate for its ills] - except, maybe, a complacent indifference to fashion, because people no longer seeking employment or promotion have less to fear."
This point suggests that the use by scholarly journals of blind refereeing is a mistaken policy. It may cause them to turn down unconventional work to which they would rightly have given the benefit of doubt had they known that the author was not a neophyte or eccentric.
(From Richard A. Posner, "Aging and Old Age")
If this hypothesis holds (and Posner admits it hasn't been tested, at least at the time of writing), then blind review may actually slow down the acceptance of theories which are radical but true. Looking up the Peter Messeri reference gave me the article "Age Differences in the Reception of New Scientific Theories: The Case of Plate Tectonics Theory". It notes:
Young scientists may adopt a new theory before their elders in part because they are better informed about current research in a broader range of fields. As Zuckerman and Merton observe with respect to life-course differences in opportunities for discovery, young scientists - closer in time to their formal training with an 'aggregate of specialists at work on [many] research front[s]' - are likely to be more up to date 'in a wider variety of fields than their older and more specialized' colleagues. Moreover, defects in existing knowledge may be more apparent to young scientists burdened with fewer preconceptions reinforced by prior practice. Conversly, older scientists may take longer to be won over to a new theoretical perspective because of greater familiarity with past successes of established theory in overcoming previous theoretical or empirical challenges. Commitment to existing knowledge may also strenghten with age because older scientists have a greater social and cognitive investment in its perpetuation, and have less to gain from adopting new ideas. This may be particularly important when adoption of a new theory requires substantial effort to master new research skills and concepts.
In this light, an older scientist's acceptance of a new, radical hypothesis should tell us to give the new hypothesis extra weight. It might even be appropriate to apply a heavier "reputation weighting" on controversial theories than established ones - we'd expect the established scientist to write papers supporting the established theories, but not controversial ones. However, blind review makes this impossible. The reviewers may even mistake the established scientist as an undiscriminating free thinker, who endorses any controversial theories simply because they're unpopular. This will slow down the acceptance of hypotheses which are, in fact, correct.
A possible objection to this could be that old scientists are unlikely to change their minds, and therefore old scientists not getting enough credit for their achievements won't have much of an effect in the spread of new hypotheses. (After all, if no old scientist endorses controversial hypotheses, then it doesn't matter if those endorsements aren't properly weighted in review.) Not so. Messeri finds that science does actually progress a lot faster than "one funeral at a time" (as Max Planck put it), and old scientists are ready to adopt new theories given sufficient evidence. While the last holdouts for outdated theories do tend to be the old, their increased security also makes them the first who are willing to publicly support new theories:
...the episode which promted Planck's 'observation' ... seems like a poor illustration of the 'fact' Planck claims to have learned. ... Wilhelm Ostwald, one of the leaders of the opposition of 'Energetics' school prominently mentioned by Planck, was only five years older than Planck, whereas Ludwig Boltzmann, whose theoretical work on entropy, in no small measure ... helped bring the scientific community around to Planck's view, was fourteen years Planck's senior. ...
In a study of the Chemical Revolution, McCann reports a negative correlation between author's age and the use of the oxygen paradigm in scientific papers written between 1760 and 1795. On closer inspection of the data, he finds the earliest group of converts to the oxygen paradigm (between 1772 and 1777) were middle-aged men with close ties to Lavoisier; the inverse age effect became manifest only after 1785, during the ten-year period of 'major conversion and consolidation'. ...
As for evolutionary theory, Hull and his colleagues find weak support for 'Planck's Principle' among nineteenth-century British scientists. The small minority of scientists who held out against the theory after 1869 were, on average, almost ten years older than earlier adopters. Age in 1859 (the year the Origin of Species was published) was unrelated, however, to speed of acceptance for the great majority of those converting to evolutionary theory by 1869. ...
I examined the publications of ninety-six North American earth scientists actively engaged in pertinent research during the 1960s and 1970s. ... The dependent variable for this study is the year in which a scientist decided to adopt the mobilist programme of research rather than to continue working within a stabilist programme. ... Before 1966, when prevailing scientific opinion still ran strongly against the mobilist perspective, the small number of scientists adopting the programme were considerably older (in terms of career age) than other scientists active during this early period. Thus, scientists adopting the programme through 1963 were on average nineteen years 'older' than non-adopters. ... Adopters in 1964 were twenty-three years older than non-adopters. ... Only with the shift in scientific opinion favourable to mobilist concepts beginning in 1966, do we start to see a progressive narrowing, and then reversal, in the age differentials between adopters and non-adopters.