Opiniomics

bioinformatics, genomes, biology etc. "I don't mean to sound angry and cynical, but I am, so that's how it comes across"

Perhaps there is a chance to change something?

OK, so here is an idea.  A few of my fellow tweeps will be aware that Heng Li’s paper on BWA-MEM was rejected from Bioinformatics;  he wrote about his experience in an e-mail and a copy of that is here.  At present, I honestly have no idea if the reasons are justified or not.

I absolutely, 100% believe in the peer review process, and would never try and circumvent it.  However, I think we all can agree that, at times, it goes wrong.  I’m not saying that the BWA-MEM rejection is an instance of this; the reviewers could be fully justified, we’ll never know as the reviews remain hidden.

As you all know, I am also a huge supporter of open peer review – I think reviewers should be identified and I think reviews should be published.

So here is an idea about how we might be able to make a difference.  Here is a potential plan:

  1. A bunch of volunteers read and digest the BWA-MEM paper in detail (it’s here).  Note: those volunteers could (and perhaps should?) include the original reviewers.
  2. Using a platform such as PubPeer.com, those volunteers review the paper fully, as if they were actually reviewing for Bioinformatics.  These reviews would be signed and fully open
  3. The full review cycle is carried out in public, with Heng responding to comments and posting the updated paper online.
  4. If the final decision is that the paper should be accepted, we petition Bioinformatics and provide them with the evidence.  Alternatively, we invite a range of journals!
  5. If the final decision is rejection, then nothing is lost and Heng at least has some good feedback

What do you think?  Imagine if this happened and Bioinformatics decided to accept the public reviews?  Wouldn’t that be a real paradigm shift?

9 Comments

  1. In itself a good idea, however, you should make sure Heng Li wants to go for it – I got the impression he is done with the paper and process.

    An alternative platform may be Peerage of Science, http://www.peerageofscience.org/. Then journals can bid to publish the paper if it gets accepted…

  2. I sympathise with Heng, but had it been me I would have resubmitted it elsewhere such as PeerJ or PLOS One. Referees and journals often ‘get it wrong’, but sometimes getting a paper published in a different journal can end up being the right thing.

    Your idea has merit, Mick, but I genuinely can’t see Bioinformatics changing its mind. It’s an interesting experiment nonetheless, *if* you can get people on board (esp. Heng himself).

    An alternative idea is for Heng to submit the paper to F1000 Research. The advantage here is that it already has the infrastructure for post-publication review, which is essentially what you’re suggesting here, I feel.

  3. Personally, I think it would be a more forceful statement, and therefore more likely to change the anonymous review system, if Heng didn’t submit the paper anywhere else and keeps the version of record at arXiv. We can still publicly peer review it at PubPeer or Haldane’s Sieve, showing that the traditional journal based peer review and validation system isn’t needed.

  4. Good comments guys 🙂

    It was a half-baked idea I had last night, I just think it is an opportunity for the community to show the real value in open peer-review, compared and contrasted with the closed system that has produced Bioinformatics’ rejection.

  5. As @ctitusbrown points out, I think this idea is already described in “Selected-papers Network” http://www.frontiersin.org/Computational_Neuroscience/10.3389/fncom.2012.00001/full
    I also like @caseybergman’s idea.

  6. Feel free to email us at Haldane’s Sieve (http://haldanessieve.org/contact/) with reviews of the paper if you would like them to be posted as stand-alone entries on the blog. Or just comment on the paper here (http://haldanessieve.org/2013/03/19/aligning-sequence-reads-clone-sequences-and-assembly-contigs-with-bwa-mem/)

  7. I also invite you to have a look at our author-guided open peer review initiative here: http://www.openscholar.org.uk

    LIBRE will launch this October and will be a free, multidisciplinary platform enabling author-guided peer review independently of journal submissions.

    Our community is always open to new members who envision a new future of open and transparent scholarly communication and want to become part of the change. Join us for free here: http://www.openscholar.org.uk/members

  8. lh3 replied to the review you all made.
    (https://github.com/lh3/mem-paper/blob/master/response_to_aaron.md)

    Nice to see this open review process ongoing. 🙂
    (perhaps this is the way the review process will be like in the future??)

Leave a Reply

© 2017 Opiniomics

Theme by Anders NorenUp ↑