Ben Mudrak: A welcome look at the research process

January 30, 2014 

Your recent two-part series “Bad chemistry” effectively highlighted some faults of the research and publication processes. At its core, science should be self-correcting; no published study is perfect, and later publications can refine or even refute prior work. This continual revision process is a fundamental part of academic research, and without the capacity to adapt to new evidence, science loses its value for creating a deeper understanding of our world.

Unfortunately, your series showed that there is not always strong incentive to set the record straight. Few journals will accept studies that merely replicate prior research, even though reproducibility is the ultimate measure of research validity. Articles that refute previously published results are typically met with increased scrutiny and foot-dragging by journal editors. Retraction is a last resort.

Sadly, there is still a stigma that makes researchers reluctant to retract a paper, even for an honest mistake like an unindexed microscope. Moreover, some journals do not want to admit that they accepted a flawed study, especially if scientists are citing the piece, which leads to increased prestige for the journal.

The scenario that you described becomes even worse when you factor in the large bias for publishing positive results. Journals and researchers prefer to see results that show a new effect, prioritizing their publication greatly over negative results. Combined with the frequent use of less-than-adequate statistical thresholds for significance, some have even argued that most published results are actually false.

While it is unclear that the situation is dire, published positive results often remain unquestioned for some time. As I can say from experience as a researcher, when other scientists fail to replicate a study, they tend to assume their replication efforts were flawed. This assumption leads to a cycle where positive results remain unchallenged.

Although academic publishing changes at a glacial pace, there are signs that researchers and publishers want to change the status quo. Researchers can now comment publicly on publications through services like PubMed Commons without having to publish peer-reviewed rebuttals. Journals such as BMC Psychology, F1000 Research and PLOS ONE welcome any sound research, including confirmatory studies. Clinical trials, which can directly affect patient lives, must now be registered ahead of time so that the results are published whether or not they are favorable for a new drug or treatment.

The launch of the Reproducibility Initiative in 2012 also provides researchers a system to have results independently validated, strengthening the manuscript they publish based on that work. Perhaps more importantly, graduate students and postdocs at universities like Duke, UNC and N.C. State are receiving more ethics training through programs like the NIH’s Responsible Conduct of Research.

With help in the form of education and new outlets for confirmatory studies or negative results, future scientists will be able to advance discovery more rapidly, even when advancement involves correcting a prior error.

Ben Mudrak, Ph.D.

Education Program Manager, Research Square

Durham

The length limit was waived to permit a fuller response to the series.

News & Observer is pleased to provide this opportunity to share information, experiences and observations about what's in the news. Some of the comments may be reprinted elsewhere in the site or in the newspaper. We encourage lively, open debate on the issues of the day, and ask that you refrain from profanity, hate speech, personal comments and remarks that are off point. Thank you for taking the time to offer your thoughts.

Commenting FAQs | Terms of Service