Citation: Kulstad E (2007) Training and Experience of Peer Reviewers: An Additional Variable to Consider. PLoS Med 4(3): e143. https://doi.org/10.1371/journal.pmed.0040143
Published: March 27, 2007
Copyright: © 2007 Erik Kulstad. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The author received no specific funding for this article.
Competing interests: The author has declared that no competing interests exist.
I read with interest your article [1] on peer reviewers' review quality and the relationship to previous training and experience, particularly the fact that your study found no easily identifiable types of formal training or experience that predict reviewer performance. As you conclude, without a better understanding of the skills in scientific peer review, journals and editors will have difficulty in systematically improving their selection of reviewers.
I wonder if an additional variable not examined in your study may prove potentially predictive of performance, namely the time committed by a reviewer to the review process in general, or a given review in particular. This data point could easily be provided by a reviewer, albeit with the caution that a self-reported number will have some subjectivity that is immeasurable. This variable could be specified either as the a priori time that an individual reviewer is willing and/or able to put towards completing a review or a self-reported time spent in actually completing a review. Either a simple dichotomized variable, say, less than four hours or greater than four hours spent on a review, or a measurement on a continuous scale, might be revealing.
Perhaps if the self-reported time commitment to a given manuscript review is found to correlate with quality of the review in a univariable or multivariable model, an additional criteria for selecting reviewers can be based on this question. A positive correlation might also explain the paradoxical findings of worse performance when being a peer reviewer for another journal or when serving on an institutional review board. Increased availability of time to commit to the process may also explain the findings of improved review quality with younger training status, if one presumes that free time diminishes with age!