Advertisement
Editorial

Better Reporting of Scientific Studies: Why It Matters

  • Published: August 27, 2013
  • DOI: 10.1371/journal.pmed.1001504

To coincide with the Seventh International Congress on Peer Review and Biomedical Publication to be held in Chicago from September 8 to 10, 2013 [1], PLOS Medicine is launching a new Reporting Guidelines Collection [2], an open access collection of reporting guidelines, commentary, and related research on guidelines from across PLOS journals. This collection is consistent with the goals of the Peer Review Congress: “to improve the quality and credibility of scientific peer review and publication and to help advance the efficiency, effectiveness, and equitability of the dissemination of biomedical information throughout the world” [2].

As early as 1990, Iain Chalmers, one of the founders of the Cochrane Collaboration, stated that, “Failure to publish an adequate account of a well-designed clinical trial is a form of scientific misconduct that can lead those caring for patients to make inappropriate treatment decisions.” [3]. Guidelines and checklists for reporting scientific studies are not just tick box exercises; rather, they help to improve the transparency and presentation of studies and, therefore, have the potential to improve the impact and implementation of scientific research.

PLOS Medicine has a strong history of promoting policies that aim to improve study design and transparency of reporting and publishing them in an open-access venue. We published our first reporting guideline – the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) Statement [4],[5] –more than 5 years ago. While the STROBE Statement was published concurrently with several other leading medical journals, critically, PLOS Medicine was the only open access journal to publish it at that time. For reporting guidelines to be useful, it is essential that they be widely disseminated, made freely available, and without restrictions on reuse. Since we published the STROBE Statement in 2007, there has been a shift toward making reporting guidelines more freely available; the EQUATOR (Enhancing the Quality and Transparency of Health Research; http://www.equator-network.org/) Network, launched in June 2008, provides freely accessible links to published guidelines.

To support PLOS Medicine's aim of encouraging the highest possible standards in medical research and reporting, the journal launched “Guidelines and Guidance” in 2008, a new section within the Magazine that publishes reporting guidelines, research priorities, methodological issues, and other articles providing guidance on the conduct and reporting of research [6].

Reporting guidelines have evolved since the original CONSORT Statement was published in 1996 [7] as a minimum set of recommendations for reporting randomized controlled trials (RCT). The CONSORT Statement was updated in 2001 and 2010, and several extensions of the guidelines have been developed based on more specific study designs (e.g., CONSORT Statement for cluster-based RCTs [8]) or specific intervention types (e.g., acupuncture [9]). While RCTs provide the strongest evidence for clinical efficacy of interventions in a clinical setting and play a critical role in healthcare decision-making, they are not always feasible or ethical to conduct. Over time, reporting guidelines have been published for many other types of research that can also influence policy and practice, such as epidemiologic [4],[5], diagnostic [10], prognostic [11], and genetic risk prediction [12] studies. Similarly, extensions of the STROBE Statement have been developed as research fields emerge, such as for use by researchers conducting genetic association studies [13] or studies in molecular epidemiology [14].

An important development in evidence-based medicine has been the use of systematic reviews to synthesize the best quality research evidence relevant to a particular topic. One of the most frequently accessed and cited papers published in PLOS Medicine is the PRISMA Statement (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [15],[16], an evidence-based, minimum set of items for reporting of systematic reviews and meta-analyses. The PRISMA Statement has been endorsed by over 170 journals and includes a 27-item checklist and a four-phase flow diagram. On the PLOS Medicine website alone, it has over 100,000 views and has been cited 1,000 times [17].

Reporting guidelines have even been developed to improve abstract reporting for RCTs and systematic reviews, as extensions of CONSORT [18] and PRISMA [19], respectively. Abstracts are the first and often only part of an article that is read. Indeed, given that 50% of biomedical research is still behind a pay wall [20], the abstract is frequently the only part of the article that readers can access. Furthermore, about 40% of abstracts for RCTs have been shown to misrepresent or “spin” study findings [21], making it all the more critical that an abstract accurately represents the research findings.

While much of the focus of reporting guidelines has thus far been on health research, the animal research community is also developing reporting standards. The ARRIVE (Animal Research: Reporting In Vivo Experiments) guidelines were published in PLOS Biology in 2010 [22] and subsequently in 11 other journals. Recent efforts by the NC3Rs (National Centre for the Replacement, Refinement and Reduction of Animals in Research) have encouraged the adoption of the ARRIVE checklist. In an Editorial published in July 2013, PLOS Medicine announced a new requirement for the ARRIVE checklist for in vivo animal studies [23].

A growing body of evidence demonstrates improvements in the quality of reporting scientific studies associated with the publication of reporting guidelines; however, translation of the guidelines into practice remains a challenge. A systematic review, published by the Cochrane Group, observed that journal endorsement of the CONSORT Statement is associated with more complete reporting of trials in medical journals [24]. Other studies have reported improvements in the quality of reporting after publication of CONSORT guidelines for abstracts [25] and the PRISMA Statement [26]. In a randomized trial published in BMJ, conventional peer review plus review looking for missing items from reporting guidelines led to improvements in manuscript quality compared with conventional review [27]. However, studies also show that the quality of reporting overall remains suboptimal [24],[28], as not all journals endorse or enforce the use of reporting guidelines [29][31].

The EQUATOR Network website houses a comprehensive library of reporting guidelines for health research [32], of which our Collection is just a subset, as well as educational materials. The PLOS Medicine Editors strongly urge (and for specific articles types, require) authors, peer reviewers, and journal editors to use these freely available resources. Most reporting guidelines have checklists that can be submitted along with a manuscript to facilitate the peer review process by allowing editors and reviewers to quickly identify essential elements of how a study was conducted.

This new Reporting Guidelines Collection aims to highlight some of the many resources now available to facilitate the rigorous reporting of scientific studies, and to improve the presentation and evaluation of published studies. Transparency in research reporting should be integral to the dissemination of scientific research. The peer review process is a critical part of research and reporting guidelines provide a mechanism to help this process. While following reporting guidelines does not necessarily make the study better, this process does give readers the information to better judge the quality, and therefore the usefulness, of research. As online publication removes the space constraints of print, reporting should be complete and transparent, and reporting guidelines aid that process.

Author Contributions

Wrote the first draft of the manuscript: AR. Contributed to the writing of the manuscript: VB JC LC RM AR PS MW. ICMJE criteria for authorship read and met: VB JC LC RM AR PS MW. Agree with manuscript results and conclusions: VB JC LC RM AR PS MW.

References

  1. 1. International Congress on Peer Review and Biomedical Publication. Available: http://www.peerreviewcongress.org/index.​html. Accessed 17 July 2013.
  2. 2. PLOS Medicine (2013) Reporting Guidelines Collection homepage. Available: http://www.ploscollections.org/reporting​guidelines
  3. 3. Chalmers I (1990) Underreporting Research is Scientific Misconduct. JAMA 263 (10) 1405–1408. doi: 10.1001/jama.263.10.1405
  4. 4. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, et al. (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. PLoS Med 4 (10) e296 doi:10.1371/journal.pmed.0040296.
  5. 5. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, et al. (2007) Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration. PLoS Med 4 (10) e297 doi:10.1371/journal.pmed.0040297.
  6. 6. The PLoS Medicine Editors (2008) Better Reporting, Better Research: Guidelines and Guidance in PLoS Medicine. PLoS Med 5 (4) e99 doi:10.1371/journal.pmed.0050099.
  7. 7. Begg C, Cho M, Eastwood S, et al. (1996) Improving the Quality of Reporting of Randomized Controlled Trials. The CONSORT Statement. JAMA 276: 637–639. doi: 10.1001/jama.1996.03540080059030
  8. 8. Campbell MK, Elbourne DR, Altman DG (2004) CONSORT Statement: Extension to Cluster Randomised Trials. BMJ 328 (7441) 702–708. doi: 10.1136/bmj.328.7441.702
  9. 9. MacPherson H, Altman DG, Hammerschlag R, Youping L, Taixiang W, et al. (2010) Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): Extending the CONSORT Statement. PLoS Med 7 (6) e1000261 doi:10.1371/journal.pmed.1000261.
  10. 10. STAndards for the Reporting of Diagnostic Accuracy Studies. Available: http://www.stard-statement.org/. Accessed 17 July 2013.
  11. 11. Altman DG, McShane LM, Sauerbrei W, Taube SE (2012) Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): Explanation and Elaboration. PLoS Med 9 (5) e1001216 doi:10.1371/journal.pmed.1001216.
  12. 12. Janssens ACJW, Ioannidis JPA, van Duijn CM, Little J, Khoury MJ, et al. (2011) Strengthening the Reporting of Genetic Risk Prediction Studies: The GRIPS Statement. PLoS Med 8 (3) e1000420 doi:10.1371/journal.pmed.1000420.
  13. 13. Little J, Higgins JP, Ioannidis JP, Moher D, Gagnon F, et al. (2009) STrengthening the REporting of Genetic Association Studies (STREGA)— An Extension of the STROBE Statement. PLoS Med 6 (2) e1000022 doi:10.1371/journal.pmed.1000022.
  14. 14. Gallo V, Egger M, McCormack V, Farmer PB, Ioannidis JPA, et al. (2011) STrengthening the Reporting of OBservational studies in Epidemiology – Molecular Epidemiology (STROBE-ME): An Extension of the STROBE Statement. PLoS Med 8 (10) e1001117 doi:10.1371/journal.pmed.1001117.
  15. 15. Moher D, Liberati A, Tetzlaff J, Altman DG (2009) The PRISMA Group (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6 (7) e1000097 doi:10.1371/journal.pmed.1000097.
  16. 16. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, et al. (2009) The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med 6 (7) e1000100 doi:10.1371/journal.pmed.1000100.
  17. 17. PLOS Reports: Article Level Metrics. Available: http://almreports.plos.org/reports/visua​lizations/8859. Accessed 17 July 2013.
  18. 18. Hopewell S, Clarke M, Moher D, Wager E, Middleton P, et al. (2008) CONSORT for Reporting Randomized Controlled Trials in Journal and Conference Abstracts: Explanation and Elaboration. PLoS Med 5 (1) e20 doi:10.1371/journal.pmed.0050020.
  19. 19. Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, et al. (2013) PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts. PLoS Med 10 (4) e1001419 doi:10.1371/journal.pmed.1001419.
  20. 20. Kurata K, Morioka T, Yokoi K, Matsubayashi M (2013) Remarkable Growth of Open Access in the Biomedical Field: Analysis of PubMed Articles from 2006 to 2010. PLoS ONE 8 (5) e60925 doi:10.1371/journal.pone.0060925.
  21. 21. Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, et al. (2013) PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts. PLoS Med 10 (4) e1001419 doi:10.1371/journal.pmed.1001419.
  22. 22. Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG (2010) Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research. PLoS Biol 8 (6) e1000412 doi:10.1371/journal.pbio.1000412.
  23. 23. The PLOS Medicine Editors (2013) Translating Translational Research into Global Health Gains. PLoS Med 10 (7) e1001493 doi:10.1371/journal.pmed.1001493.
  24. 24. Turner L, Shamseer L, Altman DG, Weeks L, Peters J, et al. (2012) Consolidated Standards of Reporting Trials (CONSORT) and the Completeness of Reporting of Randomised Controlled Trials (RCTs) Published in Medical Journals. Cochrane Database Syst Rev Nov 14;11: MR000030 doi: 10.1002/14651858.MR000030.pub2.
  25. 25. Can OS, Yilmaz AA, Hasdogan M, Alkaya F, Turhan SC, et al. (2011) Has the Quality of Abstracts for Randomised Controlled Trials Improved Since the Release of Consolidated Standards of Reporting Trial Guideline for Abstract Reporting? A Survey of Four High-profile Anaesthesia Journals. Eur J Anaesthesiol 28 (7) 485–492 doi: 10.1097/EJA.0b013e32833fb96f.
  26. 26. Tunis AS, McInnes MD, Hanna R, Esmail K (2013) Association of Study Quality with Completeness of Reporting: Have Completeness of Reporting and Quality of Systematic Reviews and Meta-Analyses in Major Radiology Journals Changed Since Publication of the PRISMA Statement? Radiology Jul 3 [Epub ahead of print]. doi: 10.1148/radiol.13130273
  27. 27. Cobo E, Cortés J, Ribera JM, Cardellach F, Selva-O'Callaghan A, et al. (2011) Effect of Using Reporting Guidelines During Peer Review on Quality of Final Manuscripts Submitted to a Biomedical Journal: Masked Randomised Trial. BMJ Nov 22;343: d6783 doi: 10.1136/bmj.d6783.
  28. 28. Ivers NM, Taljaard M, Dixon S, Bennett C, McRae A, et al. (2011) Impact of CONSORT Extension for Cluster Randomised Trials on Quality of Reporting and Study Methodology: Review of Random Sample of 300 Trials, 2000–8. BMJ Sep 26;343: d5886 doi: 10.1136/bmj.d5886.
  29. 29. Li X-q, Tao K-m, Zhou Q-h, Moher D, Chen H-y, et al. (2012) Endorsement of the CONSORT Statement by High-Impact Medical Journals in China: A Survey of Instructions for Authors and Published Papers. PLoS ONE 7 (2) e30683 doi:10.1371/journal.pone.0030683.
  30. 30. Hirst A, Altman DG (2012) Are Peer Reviewers Encouraged to Use Reporting Guidelines? A Survey of 116 Health Research Journals. PLoS ONE 7 (4) e35621 doi:10.1371/journal.pone.0035621.
  31. 31. Kilkenny C, Parsons N, Kadyszewski E, Festing MFW, Cuthill IC, et al. (2009) Survey of the Quality of Experimental Design, Statistical Analysis and Reporting of Research Using Animals. PLoS ONE 4 (11) e7824 doi:10.1371/journal.pone.0007824.
  32. 32. EQUATOR Network Library for Health Research Reporting. Available: http://www.equator-network.org/resource-​centre/library-of-health-research-report​ing/. Accessed 17 July 2013.