bioinformatics, genomes, biology etc. "I don't mean to sound angry and cynical, but I am, so that's how it comes across"

Dear NEJM…


After your terribly judged piece on research parasites, it was inevitable that you would have to apologize, though a retraction would have been better.

Here’s the thing – you came so close!  You came so close to making your apology meaningful!  Paragraphs one and three are just perfect and I cannot find a single thing I disagree with.  Those paragraphs are great.  The one in the middle?  Not so great…

Here is a reminder of what you said:

In the process of formulating our policy, we spoke to clinical trialists around the world. Many were concerned that data sharing would require them to commit scarce resources with little direct benefit. Some of them spoke pejoratively in describing data scientists who analyze the data of others.3 To make data sharing successful, it is important to acknowledge and air those concerns.3In our view, however, researchers who analyze data collected by others can substantially improve human health.

First of all, there was no indication in the original piece that when you referred to data scientists as “research parasites” that you were simply reflecting the concerns of others.  However, what’s really wrong with the above paragraph is that you do not say that those concerns should be tackled, that they should be rebutted, with evidence where appropriate; no – you simply say they should be acknowledged and aired.  Well let me just say that there are a lot of pejorative opinions expressed by humanity, on the internet and otherwise, and a large number of those definitely do not deserve to be acknowledged or aired.

If you genuinely feel, as you state, that data scientists make a valuable contribution, then you need to go back to those clinical trialists, the ones with the pejorative opinions, and say “I acknowledge your opinion, but you are wrong”.

Instead, you wrote an editorial that justified them.  Please make this right



1 Comment

  1. Another reply by Longo and Drazen continues to get it wrong:
    “In the circumstance in which data are not used in collaboration with the data gatherers, we think that reanalysts should, at a minimum, demonstrate that they can obtain the same primary findings that the data collectors published.” [ref1]

    The fallacy here is the idea that the scientists who perform an experiment are the most qualified to perform complex data analyses. All too frequently, analyses are massaged to fit a narrative, conform to a publication bias, or support a funding priority. Keeping data secret makes it easier to conceal biases–and outright fraud. The burden of reproducibility lies with the publishing author, not fact-checkers. At a minimum, NEJM should require enough data to show that their published findings can in fact be reproduced.

    Mandatory Markdown documents with source data would be a good start, although a lot needs to change before that can happen. For example, NIH policies prevent us from distributing TCGA data; scientists must request access, or download the publicly available tiers, directly from repositories. Some journals now allow us to publish Markdowns documenting our code and outputs, but we are still missing an important gap: if you want to reproduce our parasitic reanalysis, you still have to download the code and source data from different places. Those who don’t perform extensive data analyses may not realize that the insurmountable barrier to reproducing scientific findings is often the skills required to obtain, reformat, and manipulate datasets.

    None of the scientific concerns raised in the original editorial are improved by keeping data secret. Fields of epidemiology and clinical trials could learn a great deal from the experience of molecular biology, where the standard for funding and publication requires making microarray and NGS datasets publicly available. It’s arguably even more important to get this right in clinical fields, where clinical practice guidelines are often based on meta-analysis of studies that span institutions, demographics, and clinical practice patterns.

    For example, there was an outstanding question about surgical margins (should surgeons reoperate if they cut through the tumor, or if they merely cut close to the tumor). In response, the Society of Surgical Oncology and the American Society of Surgical Oncology created a joint task force. Their first objective was to commission a meta-analysis upon which they could build clinical guidelines. Longo and Drazen’s opening argument is that someone analyzing a published work will not understand the choices made in selecting the parameters. In this case, the scientists were commissioned because they were well respected experts. Fears about a “new class of research person” are trivial when the primary limitations to distilling published data into actionable guidelines are actually the lack of details and patient level data. Moreover, this is how scientific insights progress into patient care. It is hardly parasitic. And it need not be feared.

    Sharing patient-level data will generate deeper insights and more specific recommendations, with immediate patient benefit. We should all be outraged when the meta-analyses that drive our medical guidelines are based on weaker summaries of study-level data because editors allow academics to horde patient-level data for themselves.

    The aspect of this debate that I personally find the most disturbing is that prestigious medical editors joined Donald Trump in making pejorative statements about fact checkers. Scientific progress has a long tradition of challenging, and then building on, existing work. When did it become parasitic to build on the work of others? And how can it be acceptable to insult those who want to build on existing work? It’s time to tare down the misguided walls that obscure accountability and block scientific findings from impacting patients.

    Ref1: More on Data Sharing. (May 12, 2016) http://www.nejm.org/doi/full/10.1056/NEJMc1602586

Leave a Reply

© 2018 Opiniomics

Theme by Anders NorenUp ↑