Opiniomics

bioinformatics, genomes, biology etc. "I don't mean to sound angry and cynical, but I am, so that's how it comes across"

Why I resigned as PLOS ONE academic editor

I served as a PLOS ONE academic editor for roughly two years and handled a variety of papers.  This all worked very well and I helped a range of papers of varying quality and importance get published in that time period – crucially all of them meeting the publication criteria.  All good.

The problem started when I submitted a paper from my group, one which I know from experience meets all of the PLOS ONE publication criteria.  Here is what happened.

  1. 24/10/2012 – paper submitted
  2. 06/12/2012 – paper rejected based on a single 155 word review, from a reviewer who seemed to miss the point
  3. 07/12/2012 – appeal submitted by me including 917 word rebuttal letter
  4. 10/01/2013 – paper finally enters the appeals process after journal admits original academic editor has not responded to emails
  5. 07/02/2013 – the PLOS ONE appeals process uses two academic editors. After being chased up, the journal admits one of the editors has sumitted comments but the other hasn’t
  6. 04/03/2013 – again after following up, the journal tells me the decision is to send the paper out for review
  7. 04/03/2013 – I retract the paper and resign as academic editor

My main issues were these:

  1. 6 weeks to get from submission to a single, short, poor review is not acceptable for what PLOS are trying to achieve. I’m sorry but a paper deserves more than 155 words as a review.
  2. How does it take 4 weeks for PLOS to decide to start the review process? (see point 3 below)
  3. How is it acceptable for the original academic editor to simply ignore requests by PLOS for a response?
  4. Why did it take almost two months for two academic editors to decide that they need more reviews?! I don’t mind more reviews, but why take 2 months to decide they are needed?

The whole process was a farce, and ultimately, it confirmed to me that PLOS actually have no control over the editorial process at PLOS ONE, which leads to completely different standards being applied by different editors and reviewers. It also can lead to cases like mine, which PLOS even admitted was completely unacceptable.

This may read like sour grapes, but our paper was published in a different journal and we are happy with that. What made me angry was the shoddy way in which our paper was treated. I’m currently editor for a range of open access journals that I still believe in – I just don’t believe in PLOS ONE any more.

29 Comments

  1. You know similar things happen to lowly postdocs. Took me 8 weeks to be assigned an editor. A process that other journals have done in literally 5 mins. PLoS strikes me a very inconsistent.

  2. That’s certainly not good, but I can see all of the pieces of it happening. Here’s my sympathetic-to-PLOS thoughts:

    1. 6 weeks to get from submission to a single, short, poor review is not acceptable for what PLOS are trying to achieve. I’m sorry but a paper deserves more than 155 words as a review.

    I agree, but because it doesn’t push to publish exciting important work, PLOS will struggle to find reviewers so I can see this happening. A lot of that time could be taken up trying to find reviewers: waiting for them to decline to review.

    2. How does it take 4 weeks for PLOS to decide to start the review process? (see point 3 below)

    It looks like they were asking the original AE first, and had to run out of patience.

    3. How is it acceptable for the original academic editor to simply ignore requests by PLOS for a response?

    I’m sure Christmas didn’t help! There may have been personal reasons too.

    4. Why did it take almost two months for two academic editors to decide that they need more reviews?! I don’t mind more reviews, but why take 2 months to decide they are needed?

    They were clearly waiting for one of the AEs, who didn’t respond.

    I’ve not been involved as an AE of PLOS One, so I don’t know the internal dynamics, but are they suffering from being too big, and too difficult to manage? There are 4702 associate editors, and 38 section editors. So even in a strict hierarchy each section editor would have over 120 AEs beneath them. I think it would be difficult to keep track.

  3. Indeed, I can see many of the same reasons as yourself; but remember I am coming from the position of an academic editor myself. I know what standards I set myself, I know what I would and wouldn’t accept from reviewers, and I know what turnaround times I would deliver.

    For example, during the same time period I describe above, I acted as academic editor for an appeal of this paper: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0067221

    Along with another editor, we found reviewers, invited them, did two rounds of peer review with the authors and decided on publication – the result was a paper that is published and has had 91,000+ views.

    The vast majority of the appeals process was carried out within the dates I describe above.

    I realize people are busy, but I’m busy too. If you don’t have time to devote to the academic process, don’t sign up – simple 🙂

  4. Here’s something that PLOS could and should do to mitigate this kind of frustration: communicate.

    At every step in the appeal process — and indeed the initial review process — an automated email should be sent to the author. So for the initial submission:

    1. Your paper has been assigned an academic editor.
    2. Your paper has been sent out to a reviewer.
    3. An invited reviewer has declined to review; we will try another.
    4. An invited reviewer failed to accept or decline within two weeks; we will try another.
    5. A review has been submitted.
    6. A reviewer has failed to submit his report within four weeks; we are making contact again to ask for a quick response.
    7. A reviewer has failed to submit his report within six weeks; we have dropped that reviewer from this process and will try another.
    8. All reviews are in; the editor is considering the decision.
    9. Decision letter.

    And for the appeal:

    1. Your appeal has been noted and is under consideration.
    2. We have contacted the original handling editor.
    3. The original handling editor has responded.
    4. The original handling editor has failed to respond after four weeks; we are escalating to a senior editor.
    5. (perhaps) go back into some of all of the submission process.
    6. Decision letter.

    Most if not all of these stages in the process already have workflow logic in the manuscript-handing system. There is no reason not to send the poor author emails when they happen — it’s no extra work for the editor or reviewers.

    Speaking as the veteran of plenty of long-drawn-out silences from journals that I’ve submitted to, I know that getting these messages would have made a big difference to me.

  5. PLOS ONE has this annoying one-email-address-for-all system, which for me has lead to repeated cases of communication not being read by the person who had communicated to me and should know my answer. That’s another case of shoddy communication, and the main reason I did not apply to become a PLOS ONE editor when I was invited.

  6. well, why should I BOTHER talking to them? If they get eclipsed by PeerJ one day, because PeerJ improves while PLOS stagnates, so be it! I dislike PLOS ONE for a number of thing; they are very close to being perfect but don’t even want to bother getting there. Do you really think I have time to teach them to think?

  7. Yes, that’s why you should support your peer edited (!), learned-Society driven journals such as Heredity, Genetics and G3. P£O$ On€ is a money spinner for the rest of the Plos family.
    COI: I serve on the Genetics Society Executive and have editorial roles in those 3 journals I mentioned.

  8. I am sorry to learn your experience. I too had almost the same sinking experience for one of my paper, but end result is entirely different from yours. And I thank PLOS ONE editorial system for appeal as well as Damian Pattinson for ensuring a fare decision. My one recent article presented a totally novel thing, never tried before. Any new theory and proposal is destined to face tough resistance from so called big shots sitting in the field for long time. Yet, it went more dirtier as the Associate Editor handling the paper appeared to be an extremely biased person, who arranged two reviewers. The paper was rejected where more favor was given to one reviewer and the handling editor was singing the same tune. The kind of comments were showing that the reviewer was not even able to understand the work, raised wrong points which were not even in the MS, made false claims to reject the paper and the handling editor blindly(or knowingly?) accepted his claim without cross examining the claims. It looked as if authors are left on the mercy and appeasement of editors and reviewers, who become God and even stoop to any extent to screw your hard and good work. Believe me, it was a review by the reviewer and the editor, full of prejudice and clear attempt to block the work, where the points raised were totally baseless and highly objectionable. The handling editor is a big name in Roslin Institute of University of Edinburgh, working in the area of NGS ( God knows how such people are given responsible positions!). I raised the rebuttal and pointed out how entire review process appears to be wrongly done and skewed, PLOS contacted the AE. The handling editor said he was no mroe available and went absconding without any sense of responsibility and courage to give counter argument for my points. The paper was then reviewed by a Senior Editor at PLOS, as well as Damian Pattinson also got involved directly to make sure that timely steps are taken, as already long delay and loss had happened to our work. Finally, the senior editor came with totally neutral decision and accepted my claims and found the statements and claims made by the reviewer misleading.
    Without any delay, my paper was sent for re-review, and ensured that this time some highly credible names reviewed the paper ( some reviewers had disclosed their identity and were the pioners of the field). Timely review was done and with some rightly pointed concerns paper was accepted with minor revision.
    I tell you one thing. If you go to most of the journals like those of Oxford or some high headed journals who claim big science but end as a hub of lobbyism, PLOS ONE is a much resolved and neutral place to publish your work. I have some extremely bitter experiences with Oxford journals where my almost accepted papers were rejected after wasting months of time. PLOS is a community whose ears eyes and senses are open, is without any prejudice or at least a system to counter the reviewer and editorial abuse which is rampant in scientific community. Due to journals like PLOS ONE today science is becoming popular and encouraging. I am pretty sure that some one from PLOS ONE would have read your concern and would do something to correct it. They are not deaf and dumb like most of the big journals. They need improvement and need out support and inputs. You should have written to the senior editors and Damian, instead of leaving it in hurry. You should have contributed in helping this well intentioned journals in serving the community instead of leaving it. We all need good understanding and unbiased people in service. In general, the scientific community also needs to come out with some out of box idea to provide better review system where reviewer and editorial abuse could be curtailed and a review system for editors and reviewers should also exist. They need to be openly assessed for their decision on every paper, by the authors as well as readers. Even rejected papers need to be made available for assessment by readers. Let the people decide what is right and wrong, instead of endorsing closed door clandestine activities where chances of prejudice is always high. A fresh look into review system is essential for a better practice of science.

  9. I had a very similar experience.

    In my experience, quality of the Reviewers is usually not very good – comments really just make it so obvious that Reviewers have now idea what they are talking about..

  10. That’s exactly what happened to one of our paper submitted to PLoS One, same steps, similar time table… What a shame for a Scientific Journal.

  11. As far as my experience, I currently have two papers submitted to PLoS One and I am sad to say that my first paper was submitted in January 2014 and until now the AE didn’t find an external reviewer yet (that’s 6 months!!!!!) and the other paper was submitted on March 2014 which is also searching for external reviewers. My other colleagues had experiences which range from 1-10 months from submission to acceptance, which is pretty scary and seems to be more prominent the past two years.
    One thing is for sure. As an author who is fortunate to produce about 10 papers per year, I will definitely not submit to PLoS One again given such wide variations and inconsistencies. This makes me believe why PLoS One does not publish their processing times. I am a big supporter of open access, but not PLoS One.

  12. Thanks for the excellent post.

    As far as my experience, I currently have two papers submitted to PLoS One and I am sad to say that my first paper was submitted in January 2014 and until now the AE didn’t find an external reviewer yet (that’s 6 months!!!!!) and the other paper was submitted on March 2014 which is also searching for external reviewers. My other colleagues had experiences which range from 1-10 months from submission to acceptance, which is pretty scary and seems to be more prominent the past two years.

    One thing is for sure. As an author who is fortunate to produce about 10 papers per year, I will definitely not submit to PLoS One again given such wide variations and inconsistencies. This makes me believe why PLoS One does not publish their processing times. I am a big supporter of open access, but not PLoS One.

  13. What I understood about the review process is that it was focused on the quality of the research. I would imagine that any good piece of research would be published, even if the reviewers do not agree with the conclusions. They can find it not rigorous enough and ask for more experiments or reject it due to bad science, but not many more options.

    • This is what I’ve never understood about PLOS One: how can you truly separate the authors’ conclusions from (your opinion of) the quality of the research? If you don’t believe the authors’ conclusions, you natural assessment will be that the research wasn’t rigorous enough (if only they had done abc control or xyz other method, the effect would disappear. Thus, the method isn’t adequate to the question they’re asking.). The idea of separating quality of research from conclusions sound good, but I’m not certain I believe it can be done in real life.

  14. Plos one is just horrible. I never reference it. Terrible papers most of the time , one wonders who reviews these manuscripts

    • Just for the record: in my field (palaeontology), PLOS ONE is very well respected, and tends to carry excellent, comprehensive work that is illustrated in detail and highly useful. I don’t know what field John Goodman is in (assuming he’s not merely a troll) but evidently things are very different in paleao!

  15. I am on six months and so far nothing!

    • You’re not alone – initial submission Nov 2015 and now mid June 2016 and NOTHING, have not received a single review.

  16. Here is another example what I consider an example of the PLoS One Editorial staff ineptitude.

    This did not end well, and I believe that my treatment was completely unfair.
    Following the publication of my paper , some unsubstantiated and inappropriate comments were posted to the PLoS One website. They were taken down, I presume because some of the comments were offensive and probably personal rather than scientific. The journal stated that “some of the comments were not in line with their policies for comments”

    The original paper was published in Feb of 2011.

    These comments posted within days of the publication, resulted in with an almost two year battle with the journal, that ended in an UNFAIR ridiculous and vague expression of Concern published in Dec. of 2012 that did not contain any data or references.

    There were three requests for information, one in March 2011, a second in August of 2011, and then a third request almost a year later in June of 2012. Each time the same points by the Journal’s advisors were raised that were not substantiated with data or references. Every time an appropriate reply was sent to the journal, with references to back up the response. By Sept of 2012, when I heard back from the Journal it was my opinion that NO consideration had been given to the responses given to the journal, and was told that they were going to publish and Expression of Concern. At best this Expression of Concern was a difference of opinion. I have around 30 pages of correspondence with the journal regarding the expression of concern, which is excessive to make a simple decision that should be based on scientific facts and data.

    In September of 2012 they offered me 6 months to come up with data to refute the expression of concern. Again this request was unreasonable in terms of a scientific research timetable. I wondered why I had to provide data while the expression of concern did not contain any data only speculation. In March of 2013, I contacted the Journal stating that I had data to refute the expression of concern, however this was 7 months past September 2012, and I was told that I would have to submit another manuscript with them!

    As you can imagine THERE WAS NO WAY THAT I EVER WANTED TO DEAL WITH PLOS ONE AGAIN!!!!!!! I ended up publishing this data in Clinical Epigenetics, a much better Journal in my opinion, and the review process was fair and timely.

    In the case of PLOS ONE, the editorial staff was fumbling around for years and in my opinion were making decisions that were well outside of their authority.

    Mary Whiteley

  17. Giovanni Tarantino

    27th November 2015 at 8:56 pm

    My impression about the reviewing process of a paper recently submitted to PlosOne by my group is that the AE and mainly Reviewers have not evaluated the research with constructive criticism, have based the decision exclusively on their personal approch to the topic and for rejecting the paper have found inconsistent objections. It is a shame….. but the decrease of journal’s IF year after year is the right answer.
    Obviously, it was the last time I submitted any paper to this journal.

  18. I have TODAY just received a first request to review a paper for PLOS One. I am an emeritus professor, and an epidemiologist, and have been a reviewer for several international medical and health journals for some years. I take the role of reviewer very seriously, putting many hours into every review.
    I was surprised in this instance to see the list of ALL authors’ names submitted along with the abstract. Does PLOS One also reveal the reviewers’ names to the authors? This surprised me. I have never seen “open” reviews in a respected journal in the medical/health science field.
    I will have to decline the request in this instance because of a necessary period of travel starting Tuesday, but the lack of a ‘closed’ review protocol may persuade me to decline future requests as well.

  19. It is almost a year now. I went through a swift rejections based on 1 positive reviewer and one 50/50. There were 2 major criticisms, the lack of novelty as well as serious methodological error (C3 neuro activation metric). The novelty is not even part of PLOS ONE requirements and the metric we used we managed to defend in a rebuttal. On such formal grounds it was successful. Now they are asking to suggest editors, while sending every now and then an apologetic emails for the long delay.

    To recap, things seems to have gone from bad to worse. Reconsidering at the moment, I might give them some deadline and just move to another journal.

    • In my field it’s rare not to see all the contributors. I can remember the blind review only a few times, and in most cases an educated guess could be made who the authors were.

Leave a Reply

Your email address will not be published.

*

*

code

© 2017 Opiniomics

Theme by Anders NorenUp ↑