The one thing “Setting the Research Record Straight,” last night’s Science Online New York City event, made abundantly clear is that it’s better to catch scientific error and misconduct pre-publication than post-publication.
Fortunately, journals have tools at their disposal to catch image manipulation. Unfortunately, data manipulation is only one form of misconduct: other error or fraud cannot be detected so easily.
Liz Williams, executive editor of The Journal of Cell Biology discussed the steps the image-heavy journal takes to prevent image misconduct. The journal employs someone whose full-time job is to screen all images from accepted papers. When questions arise during the screening process, the journal requests the author’s original data. Not all manipulation is fraudulent: If image manipulation violates the journal’s guidelines but does not change the results of the research, authors are asked to remake the image.
If the image is fraudulent, changing the interpretation of the data, the journal revokes the article’s acceptance or asks the authors to withdraw the piece. The number of revoked or withdrawn papers has remained steady at one percent for the last ten years—had the journal published the articles, this would translate to four articles per year.
Post-publication, said John Krueger of the Office of Research Integrity (ORI), readers can review images and articles “in perpetuity” using readily available detection tools. Unfortunately, people challenging images and data often have not been educated in how to use these tools. “The challenge is in interpretation more than detection,” he said. Moreover, “allegations go away with data” when there is no evidence of fraud or error, but often researchers do not hold onto data post-publication. “Serious data retention policies are needed,” said Krueger.
But, as Ivan Oransky, executive editor at Reuters Health and co-author of the blog Retraction Watch, said, “We are all gatekeepers now.” Retractions are on the rise, in part because blog authors are getting more aggressive in bringing problems to the attention of journals. Journals, said Oransky, “need to do better. They need to trust outsider comments more.” And, he added, “post-publication review is the future.”
What are the reasons for scientific paper retractions? Both Oransky and Krueger pointed to an article in The Journal of Medical Ethics, in which author R. Grant Steen evaluated retracted English language papers in PubMed from 2000-2010. Of the 742 retracted papers, 73.5 percent were due to error and 26.6 percent due to fraud. The most common reasons given for retraction were scientific mistakes and data and textual plagiarism.
In certain cases, not retracting a paper can be dangerous. In the case of medical literature, said Krueger, “papers may be key components of clinical trials.” Krueger’s office once brought its concern about a paper to the FDA; it turned out the paper was the only proof-of-concept for a clinical trial. In another study, R. Grant Steen found that 70,501 patients were treated in 851 studies that cited retracted research—often research retracted for fraud.
Unfortunately, the retraction process can take a long time. Krueger explained that while ORI investigations should take 120 days, some go on for years. “It takes time to make sure the facts are right,” said Krueger, especially when researchers, institutions, and journals all play a role in the process. In the meantime, the journals have to decide at what point to comment or make an “expression of concern.” It may not be in the journal’s best interest to comment before all the facts are known.
After retractions occur, however, the research record must be corrected. Oransky pointed to a 1999 article from JAMA that is still frequently cited in support of research.
In some cases where citations of retracted articles occur, it may be because journals do not make it clear that an article has been retracted. But other journals do provide good retraction notices. In some cases, said Krueger “you can learn more from reading a retraction than from reading the original paper. Let’s not lose sight of that . . . Retractions are a healthy part of the scientific process—they are a sign that science is doing its job.”