Advertisement
Research Article

The Relationship of Previous Training and Experience of Journal Peer Reviewers to Subsequent Review Quality

  • Michael L Callaham mail,

    To whom correspondence should be addressed. E-mail: mlc@medicine.ucsf.edu

    Affiliation: Division of Emergency Medicine, University of California, San Francisco, United States of America

    X
  • John Tercier

    Affiliation: Department of Sociology, University of Lancaster, Lancaster, United Kingdom

    X
  • Published: January 30, 2007
  • DOI: 10.1371/journal.pmed.0040040

Reader Comments (4)

Post a new comment on this article

An additional model variable to consider

Posted by plosmedicine on 31 Mar 2009 at 00:05 GMT

Author: Erik Kulstad
Position: MD, MS
Institution: Advocate Christ Medical Center
E-mail: ekulst@yahoo.com
Submitted Date: February 07, 2007
Published Date: February 9, 2007
This comment was originally posted as a “Reader Response” on the publication date indicated above. All Reader Responses are now available as comments.

I read with interest your article on peer reviewers’ review quality and the relationship to previous training and experience, particularly the fact that your study found no easily identifiable types of formal training or experience that predict reviewer performance. As you conclude, without a better understanding of the skills in scientific peer review, journals and editors will have difficulty in systematically improving their selection of reviewers.

I wonder if an additional variable not examined in your study may prove potentially predictive of performance, namely the time committed by a reviewer to the review process in general, or a given review in particular. This data point could easily be provided by a reviewer, albeit with the caution that a self-reported number will have some subjectivity that is immeasurable. This variable could be specified either as the a priori time that an individual reviewer is willing and/or able to put towards completing a review or a self-reported time spent in actually completing a review. Either a simple dichotomized variable, say, less than four hours or greater than four hours spent on a review, or a measurement on a continuous scale, might be revealing.

Perhaps if the self-reported time commitment to a given manuscript review is found to correlate with quality of the review in a univariate or multivariable model, an additional criteria for selecting reviewers can be based on this question. A positive correlation might also explain the paradoxical findings of worse performance when being a peer reviewer for another journal or when serving on an IRB. Increased availability of time to commit to the process may also explain the findings of improved review quality with younger training status, if one presumes that free time diminishes with age!

Competing interests declared: Serve as peer reviewer for Annals of Emergency Medicine.