Peer Review: An Introduction and Guide

What is it? How does it work? What about misconduct? Is there an alternative?

Mark Ware

 Get the full PRC Peer Guide (27pp)

 Executive Summary

The reader of this short Guide will be left with a coherent and forward-looking overview of the processes, the shortcomings, and the innovations around peer review, and a deeper understanding of why peer review is such a vital element of effective scholarship.

Editorial peer review is the core distinguishing feature of the academic journal. The key purpose of peer review is to appraise the quality and value of a research article, usually prior to publication, but it also helps considerably to improve the published version.

The Publishing Research Consortium has commissioned Peer Review: An Introduction and Guide from Mark Ware as an overview and critical appraisal of the peer system, both as practised now and how it might evolve in the future. Ware discusses how publishing in journals is an integral part of the research process, offering diverse benefits for research funders, policy-makers, wider society, and the economy at large, as well as to researchers and the research community itself.

The strengths of current peer review processes, which are described in the Guide, include the motivation for authors to improve the quality of their submissions and the valuable feedback that can improve the published version.  Evidence from surveys suggests that 90% of researchers believe that peer review directly improved their last submitted paper.

However, peer review is not without its critics, who question the effectiveness, efficiency and fairness of the processes used. Although peer review can be regarded as a burden on researchers, it is generally viewed as an ‘appropriate burden’ that supports the research process and the professional status of researchers as academics. Nevertheless, critics suggest, potential problems associated with the process such as anonymity of reviewers, inherent bias (through nationality, gender etc.) and the time delay before publication represent key shortcomings.

Evidence for a ‘peer review crisis’ however appears largely unfounded. Journal editors are exploring both open review and double-blind review to help mitigate the natural human flaws in the process. Worldwide growth in the number of active researchers also means more potential reviewers, so greater efficiencies are possible.

Innovation in peer review is dynamic, with new approaches being developed and widely trialled across the spectrum of journals and publishers. Approaches based on ‘Soundness not significance’ have been widely adopted by publishers such as BioMed Central and PLOS. This model has come to be associated with broad-scope non-selective journals that adopt a peer review process that assesses articles for technical soundness but not for impact or significance. The model is not without issues since a ‘superjournal’ such as PLOS ONE might publish thousands of articles each year yet lack effective tools to help researchers focus on those articles that are of most relevance to them.

Other interesting new approaches are described in the Guide, such as ‘cascade’ journals (that transfer rejected articles to other relevant journals and so save resubmission by the author), and ‘portable’ peer review (where the review is conducted independently of a journal and so can accompany the article on submission).

Post-publication peer review potentially represents the most radical change from the current system. In its purest form, post-publication review would see all papers published without any prior review process, with various forms of rating and appraisal taking place subsequently. This way, it is argued, the time-consuming and cost-inefficient processes of appraisal and filtering articles before publication can be reduced. A hybrid form might complement post-publication rating for significance and impact with some form of pre-publication review based on ‘soundness not significance’. The emerging technology of ‘altmetrics’ might also be used to assess impact across the media at article level.