The Dark Side of Peer Review

“Peer review” is the process in which manuscripts describing research findings are critiqued by other scientists in the relevant field to see whether the work is of sufficiently high quality to be published.  There can be little doubt that peer review can be a useful method for intercepting inferior quality data and for offering advice on the key experiment or experiments that the research team need to perform in order to substantiate their story.

For a long time, however, there have been concerns within the scientific community that the process is open to abuse by unscrupulous scientists exploiting their role as reviewers to further their own research. In recent weeks these concerns have spilled into the general press and generated alarming headlines.

First, there was the “Climategate” scandal in which leaked e-mail exchanges within the Climate Research Unit at the University of East Anglia brought into doubt the veracity of some of the evidence for global warming. The details of the science itself are not relevant to the discussion here, and many of the original accusations against the CRU look considerably less damning in the wider context in which they were written (see, for example, the interesting video Climate change: those hacked e-mails). Nevertheless, one feature of the memos was the insinuation that peer review could be manipulated.

Added to this, we now have the spectacle of leading stem cell scientists complaining that their work has been ‘slow-tracked’ by major journals in favour of other research being carried out by rival teams (for details see the BBC report Journal stem cell work ‘blocked’ or listen to the audio Stem cell research ‘biased’ (Today programme, 2nd February 2010).

In truth, there are lots of different issues at play in this story. One is the rivalry between research teams for prestige. Coupled with this there is the rivalry for research money. As the clip from the Today programme makes clear, research funding is related to a scientists track record in publishing in major research journals, those with a high “Impact Factor” (this also has a huge bearing on research ‘league tables’ as the same criteria have been key aspects of the Research Assessment Exercise (RAE) by which the relative merit of similar departments at different Universities are compared. Publication in a major journal such as Nature or Cell can be make or break for successful receipt of the next grant, a decision which Austin Smith says may be worth half a million pounds.

Journal publication is a business venture, so editors have a vested interest – it is alleged – in trying to make sure the work published there has topicality that will boost their Impact Factor and hence their esteem, their advertising revenue, etc. Some of these pressures might be alleviated by changes in the publication model (i.e. by a shift to more open access journals) but the fundamental difficulties of personal rivalry remain a flaw in the peer review process, even if the financial implications are negated.

In the absence of other methods for evaluating the quality of research, the peer review system is unlikely to be replaced in the near future. There is, however, an onus on editorial boards, and on the reviewers themselves, to see that the greatest possible integrity is maintained in ensuring that decisions about the merits of a paper for publication are made on the right grounds. The importance of virtue ethics comes into play here, with the character of the individual being a significant determinant in their actions.

UPDATE: This link goes to the contents of the Open Letter to Senior Editors.  There was also a second interview from the Today programme, with Sir Mark Walport of the Wellcome Trust. As suggested above, he emphasises the need for quality control and the potential to put the peer review documents on-line too so that the process is more open. He suggests that the reviewers need to remain anonymous so that they do not need to pull their punches.