Opiniomics

bioinformatics, genomes, biology etc. "I don't mean to sound angry and cynical, but I am, so that's how it comes across"

Are you in peer review debt?

I had been thinking about writing a post about this for some time, then serendipitously, this tweet came up in my timeline:

Well I’ve got something to say about that, but we’ll get to that shortly.

I want to start with a gripe – as academic/associate editor at FrontiersIn and BMC (and formerly PLOS ONE), I often find it really difficult to find reviewers for papers. I’d say around 50% of the time, an invite is sent and is simply ignored – not accepted, not declined, just left to expire. I’m not sure on what planet this is an acceptable “response” to a peer review invitation, but it’s not on my planet. If you can’t do it, click “decline to review” – it really helps and prompts editors to invite someone else. Ignorance is just rude. I’m keeping a list by the way, things will go far better for you if you stay off my list πŸ˜‰

Now, let’s get to @AxiosReviews’ flawed tweet. According to the US Library of Medicine, it looks like the average paper has 5 authors, not too different to the tweet above’s claim of 4.5. The number of peer reviews is intuitively between 2 and 3, so 2.6 seems a decent estimate too – though there is no mention of the editor, who is often simply an academic donating some of their time to the journal, just like a peer reviewer.

The fatal flaw of the tweet above is the maths – the number of authors / number of reviews leading to 0.6 reviews for every paper you’re on. This assumes all authors are equal, and of course they are not. Editors look at prior publications to select reviewers and are simply not going to choose someone from the middle of a list of authors; so the effective number of authors per paper is 2 – the first author and the last author.

If we then accept 2.6 as the average number of reviewers, and add in the editor, we get 3.6.

The maths now looks quite different – for every first or last author paper you publish, you need to complete 1.8 reviews – almost double.

But wait – this is still wrong. This assumes each paper is only reviewed once, clearly untrue for papers that are rejected by certain journals before eventually being accepted, and those papers that are reviewed but never published (for the purposes of simplicity I am counting multiple rounds of review of the same paper at the same journal is one unit of peer review).

Rejection rates are hard to come by, but this article suggests that 25% of papers go first to a journal different to the one they eventually get published in. So we need 25% more input from editors and reviewers, and our 3.6 turns into 4.5.

Where we end up is that, for every first or last author paper you publish, you need to carry out 2.25 editorial or peer review activities.

We return to my question – for every first or last author paper you need to carry out at least two editorial or peer review activities – are you in peer review debt or credit?

6 Comments

  1. When I ‘ignore’ a request that usually means its a maybe, and I’m hoping to reply at the end of the week if I decide if I can squeeze it in after all.

    When I plan to say ‘no’ I may delay a day or two while pondering alternative reviewers – if the automated reply form catered to this better you’d have a more immediate ‘no’ and (hopefully) some further suggestions later on.

  2. Hi Peter,

    When I say “ignore”, I mean the whole time limit assigned to the invite has been reached and there wasn’t a response.

    How would you design the perfect reply form? Not that editors have any control over this…

    Cheers
    Mick

  3. I’d add even way more to this. I went to grad school, published 4 papers, never was asked to review anything, and am now in industry where I’m not part of the reviewer pool. I bet there are as many or more like me who may publish a few papers and leave the field.

  4. If you are (associate) editor at one or more journals. do you get a discount of your review debt? I agree with the mails that get simply ignored, not very helpful in the whole process.
    But if you are too nice you get ‘over-xploited as a referee. At some stage I seemed to be ‘catch-of-the-day’ for several BMC journals (some of you have suffered the consequences) and I feel that a justified ‘no’ is often respected esspecially if you can nominate a suitable colleague.

  5. Hi DJ

    I absolutely agree – “No” is an acceptable answer, as long as it’s because you have done more than your fair share of peer review and editorial responsibilities.

    “No” is not an acceptable answer because you are busy – we’re all busy – and anyone who thinks they are more busy than everyone else is arrogant.

    The worst answer is ignorance, by far.

    Cheers!

    Mick

  6. I guess I ought to defend my numbers a bit: the calculation comes from a piece I published a while back (http://www.nature.com/nature/journal/v468/n7327/full/4681041a.html). I was trying to explain the apparent elasticity in the supply of reviewers during a doubling of submissions to Molecular Ecology. I think we probably agree on quite a few of the ailments affecting peer review, but I don’t agree with a few of your criticisms of this calculation:

    “Editors look at prior publications to select reviewers and are simply not going to choose someone from the middle of a list of authors; so the effective number of authors per paper is 2 – the first author and the last author.”

    >My day job at Molecular Ecology means I need to find reviewers on a regular basis, and I pick from the entire author list on past papers. The first author is often a grad student and not suitable, and the last author often a senior academic and likely to decline the request. I therefore don’t agree that the effective number of authors per paper is two. It might not be as high as 4.6, but as most of the authors could and should be asked to review other papers on that topic it’s not as low as 2.

    “If we then accept 2.6 as the average number of reviewers, and add in the editor, we get 3.6.”

    >My tweet was about the number of papers you should review per submission you’re an author on, and didn’t include editorial work. Many reviewing academics aren’t editors. I concede that some broader calculation of the number of people involved in a round of peer review should contain the editor.

    “But wait – this is still wrong. This assumes each paper is only reviewed once, clearly untrue for papers that are rejected by certain journals before eventually being accepted”.

    >In the tweet I said per submission (“sub”), which entails a single round of review. A lot of papers go through multiple rounds at different journals and take up lots of reviewer time, which is why I founded Axios in the first place.

Leave a Reply

© 2017 Opiniomics

Theme by Anders NorenUp ↑