Given the central role of peer review in determining the quality and credibility of published research, establishing a rigorous evidence base for its evaluation is an important priority for the scientific community. However, current research in this area is characterized by substantial heterogeneity. Evidence from various studies, including a systematic review by Bruce et al. (2016), demonstrate wide variation in the outcomes used in peer review research, ranging from retraction rates and reviewer burden to manuscript quality. This lack of standardization impedes evidence synthesis and limits the comparability of findings across studies assessing interventions targeting peer review. Creating a core outcome set may consequently offer a structured approach to standardizing outcome selection and measurement. Accordingly, this project seeks to establish an international consensus-based core outcome set for research on manuscript peer review.
Bruce, R., Chauvin, A., Trinquart, L., Ravaud, P., & Boutron, I. (2016). Impact of interventions to improve the quality of peer review of biomedical journals: a systematic review and meta-analysis. BMC medicine, 14(1), 85. https://doi.org/10.1186/s12916-016-0631-5
Mehvish Masood (1), Ka Hin Tai (2), Julia Soto Rizzato (3), Janice Tufte (4), Michael Mittelman (5), Christine Von Raesfeld (6), Jessie McGowan (7), Alexandra Bodnaruc (8), Isabelle Boutron (9, 10), Jocalyn Clark (11), Martin Offringa (12), Ludo Waltman (13), David Moher§ (7, 8, 14)
§Principal Investigator
(1) Schulich School of Medicine & Dentistry, University of Western Ontario, London, Ontario, Canada
(2) Univ Rennes, Inserm, EHESP, Irset (Institut de recherche en santé, environnement et travail), UMR_S 1085, F-35000 Rennes, France
(3) Faculty of Pharmaceutical Sciences, Universidade Estadual de Campinas, Campinas, São Paulo, Brazil
(4) Patient Partner, Cochrane Consumer, PCORI Ambassador, Seattle, WA, USA
(5) Michael Mittelman, Patient Partner
(6) Patient Partner and Research Collaborator, AI Rights Initiative, Light Collective, Santa Clara, CA, USA
(7) School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
(8) Centre for Journalology, Methodological and Implementation Research Program, Ottawa Hospital Research Institute, Ottawa, Canada
(9) Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Centre for Research in Epidemiology and Statistics (CRESS), Paris, France
(10) Centre d’Epidémiologie Clinique, Hôpital Hôtel Dieu, AP-HP, Paris, France
(11) Department of Medicine, University of Toronto, Toronto, ON, Canada
(12) The Hospital for Sick Children, Toronto, Ontario, Canada
(13) Centre for Science and Technology Studies (CWTS), Leiden University, Leiden, The Netherlands
(14) Institute of Health Policy, Management & Evaluation, Dalla Lana School of Public Health, University of Toronto, Canada
Disease Category: Methodological & diagnostic
Disease Name: N/A
Age Range: Unknown
Sex:
Nature of Intervention: N/A
- Clinical experts
- Epidemiologists
- Journal editors
- Methodologists
- Patient/ support group representatives
- Policy makers
- Researchers
- Statisticians
- COS (Other)
- Consensus meeting
- Delphi process
- Literature review
- Survey
Guided by the COMET Handbook and COS-STAD recommendations, this program of research will adopt a multi-stage design comprising a scoping review followed by a Delphi consensus process. The scoping review will inform the development of an initial candidate item list by identifying and synthesizing existing outcome and assessment approaches reported within the literature. A subsequent three-round Delphi process will be conducted with multiple stakeholder groups (e.g., researchers, clinicians, journal editors, publishers, patient partners, and institutional representatives). The first two rounds will be administered using online surveys, during which participants will rate and provide feedback on the proposed core outcome items identified in the scoping review. Consensus will be reached if 80% of respondents score an item within the top (4 to 5) (‘include’), or bottom (1 to 2) (‘exclude’) of a 5-point Likert scale. Results from these initial rounds will be synthesized and carried forward to the third round, a one-day consensus meeting, where stakeholders will discuss and agree on the final set of core outcomes and their measurement properties. The final deliverable will be an international core outcome set for assessing manuscript peer review.