I got into a really long, and often interesting, conversation on Twitter a few days ago about the merits, or not, of open peer review. 140 characters is a bit limiting, so I am putting my arguments here.
My regular readers know that I am a big supporter of open peer review, and I have signed grant and manuscript reviews for about 2 years now – crucially, I sign them whether they are positive or negative. However, what I really want to do is change the way we see peer review – in my opinion, we should see it as a supportive and collaborative process by which a group of independent scientists assess the quality of a body of research, suggest ways in which it might be improved and decide whether it is ready for publication. Some of this is encapsulated in my reviewer’s oath. Peer review doesn’t have to be the often awful process that it often is, and it also doesn’t have to be easy on the authors.
Anyway, below are the arguments that I have read about why peer review should remain anonymous. My response is below.
“Anonymous peer review protects reviewers”
I am immediately concerned by the language here. Protect reviewers from what?! Is peer review suddenly a dangerous activity? The point is made even more forcefully in this quite terrifying piece, which states:
It might be especially difficult to find referees for authors who hold positions of power and influence, or for those who are considered quarrelsome or vindictive by their peers. In particular, younger, less-established scientists … would be reluctant to reveal themselves, for fear of retaliation from their more powerful colleagues.
OK, OK. Hands up who in science is happy with the idea that in our field “powerful” and “vindictive” scientists might want to “retaliate” against someone who has reviewed their work?!! Does anyone seriously think that that’s OK? Why are we even discussing peer review if we work in a field where this might happen? Stop the clock. Time out. Pause. Retaliatory, revenge attacks, by anyone, should be considered serious scientific misconduct and the perpetrators should be identified and sacked. It’s as bad, worse, than plagiarism or making up data. This kind of activity should spell the end of careers. Are we expected to sit back and accept that this kind of thing might happen?!
Of course, this argument in support of anonymous peer review is actually a very powerful argument against it. Powerful or vindictive scientists are only able to take revenge attacks because they can hide behind a cloak of anonymity. If their reviews were published, alongside their names, then the community would soon recognise if they were behaving badly, and action would soon follow. The collective scientific community are actually very ethical and moral, with a strong sense of fair play. Should a scientist identify retaliatory, revenge reviews, with evidence, we would all listen. And act.
“Open peer reviews are ‘soft'”
I refer you to this paper, which states:
Asking reviewers to consent to being identified to the author had no important effect on the quality of the review, the recommendation regarding publication, or the time taken to review
So in fact there is no evidence that open peer reviews are in any way “soft”. Hilariously, the Nature Neuroscience piece throws out this peer reviewed, scientific evidence in favour of “an informal poll of some of our referees”. What?! You can’t just throw away a paper’s conclusions based on a conversation you may or may not have had with a few carefully chosen referees!
There is no evidence that open peer reviews are soft, and I therefore declare this argument invalid.
“Open peer review promotes ‘quid pro quo’ favourable reviews
Yes, in fact I think open peer review probably does enable this kind of unethical, fraudulent behaviour; my simple point is that anonymous peer review doesn’t stop it either! They’re as bad as each other. There is absolutely nothing to stop an anonymous reviewer emailing the authors of the study, praising their amazing research, pointing out they were reviewer 2 and inviting them to review their future papers.
Open peer review does not solve the “favourable reviewer ring” problem; but nor does anonymous peer review!
“Anonymity removes egos from the equation”
This one actually make me laugh out loud! Sure, because as anyone who has ever been involved with anonymous peer review, egos are completely absent…..
“Open review will make it hard to find reviewers”
Yes, I think this is probably true and is backed up by the BMJ paper above. I have one simple riposte to this. When you look at a body of work, with the aim of reviewing it, the only thing you should be thinking about is the quality of the science (and perhaps also the impact, if the editor wants you to consider that). If you would be scared to reveal your name to the authors, then you are already showing evidence that you’re unable to give an objective review. You are already thinking, consciously or subconsciously, about who the authors are, what they represent to you, how they might affect you and your work. You cannot be objective and you shouldn’t review the paper, anonymously or otherwise. No review is better than a biased review.
“Anonymity corrects for unequal power relationships”
No, anonymity enables unequal power relationships, and open data, open information, open peer review equalizes those power relationships and ensures all interactions can be assessed independently for fair play.
One of the most telling quotes from the Nature Neuroscience piece is this one:
The journal also experienced an occasional breakdown of the peer-review process, in which authors and referees bypassed the editors completely in negotiating how a paper should be revised.
OH MY GOD! Stop the planet! Everyone just HOLD! This is serious! Imagine, authors and reviewers collaborating together to make a piece of work better, outside of the control of a journal! THIS MUST BE STOPPED!
(seriously, you can’t make this up!)
In case you hadn’t noticed, the BMJ has practised open peer review since 1999, and they took this decision on the basis that it was simply more ethical, without cutting down on quality. Good on them. They seem to be doing OK for themselves! If you don’t have time to read it, here are some choice quotes:
All editors have seen curt, abusive, destructive reviews and assumed that the reviewer would not have written in that way if he or she were identifiable
openness should eliminate some of the worst abuses of peer review, where reviewers—under the cloak of anonymity—steal ideas or procrastinate
Openness … links accountability with credit
Science is progressively moving away from anonymity.
we hope our small move will contribute to a broader culture change so that junior researchers cease to fear reprisals from senior ones
Inspirational stuff, I am sure you’ll agree.
What else can I add to this? Just my own experiences. Putting my name next to reviews has made me, without exception, give more thought to what I write, and that can only be a good thing. I think more about the evidence for my criticism and whether those criticisms are valid. More importantly, I empathise with the authors. This is a good thing. It doesn’t make me soft, it just helps me understand. By far and away the biggest reaction I have seen is one, not of anger, but of respect.
Detractors will say that perhaps I have had a negative reaction, I just haven’t seen it; and perhaps they are right. In fact, the only way I would see is if those reactions were open, and signed with the names of the reviewers responsible. Which makes my point very nicely.