Peer review – what is it and does it deserve our trust?

Dylan Lynch

Staff Writer

The process of peer review is one of the most widespread practices in the field of scientific publishing. Most articles or journals you have ever read would have gone through some form of peer review. Usually, it involves professionals working in the same area of science as the author evaluating the piece of work in order to maintain and enhance the quality of work produced in the field, and to make sure that it is worthy to go to publication. But why should we care about peer review, and is it really necessary at all?

The first recorded occurrence of peer review by an editor was at the Royal Society in 1665, while the first peer reviewed publication is thought to be Medical Essays and Observations, published by the Royal Society of Edinburgh in 1731. The method we use in the present day actually evolved from the 18th century process, with a few tweaks. The process is quite widespread, and is used for all papers published in high profile journals such as the journal of the American Medical Association (AMA), ‘Science’ and ‘Nature’.

So how does this 300 year old process actually work? A journal editor will send the article or paper to a group of reviewers, otherwise known as ‘referees’. These referees spend anywhere between a fortnight and a month (or even more) scrutinizing the details of the paper and begin identifying scientific errors, and questioning any methods that may have been used in the process of the experiment. The team of referees also judge the significance and originality of the work, and determine how well the paper advances the field of science. They can then recommend if the paper be published or rejected. The journal editors make the final decision but more often than not, they accept the referees’ decision. The review can be single-blind, double-blind or open. Single-blind means that the authors name is known to the reviewer, but not the other way around. Double-blind then obviously means that neither author nor referee knows each other’s identity, and open means that the identities are known to the referee and author. Many editors and research scientists have been pushing as of late for the open peer review system to be implemented. It is believed that this system, where no individual is anonymous, will open dialog between researchers and referees so that well thought-out replies and adjustments can be made to papers before publishing.

Unfortunately, the peer review process is far from perfect. The process is extremely slow and also quite expensive. When a peer reviewed paper is published, it can cost up to $5000 (¤3700) for the academic community to gain access to it.  A huge chunk of this reportedly goes as profit to the publisher, while the rest covers distribution costs, editorial costs and other expenses. Not only that, but the review process can often produce inconsistent results. If one reviewer finds an article well written and thoroughly researched but another finds it the complete opposite, then months and months of research and hundreds of pounds may have been wasted. However, the failures of peer review can be much more serious.

In September of 2012, a scientific paper was published which stated that genetically modified (GM) corn causes cancer in laboratory rats. This article caused widespread confusion in America where this particular brand of corn is quite popular. It even caused Anti-GMO (genetically modified organism) groups to storm into a GM-corn storehouse in Southern France and destroy several units of the foodstuff. In 2010, a well-respected journal reported that the University of Texas had confirmed homeopathic medicine could kill cancer cells. The problem with these papers was that the data published was altered, and the scientists only presented the data that made it seem like the accusations against GM-corn and allopathic (conventional) medicine were true, a process known as “cherry picking”. These alterations were not detected during peer review, the only barrier in place to prevent bogus data from being published.

The GM-corn study, carried out at Caen University in France, reported that huge tumours were observed in rats fed the GM-corn. However Professor of Cell Biology at Edinburgh University, Anthony Trewavas, claimed that not only was the sample space of 200 rats far too small, but he also stated “To be frank, it looks like random variation to me in a rodent line likely to develop tumours anyway”. Furthermore, only 10% of the rats were the control group (i.e. not fed GM-corn, in order to see if tumours would arise anyway) and any rats which ingested the supposedly toxic GMO outlived the rats that had a ‘clean’ diet.

The list of failed peer reviews goes on and on, from linking anti-rheumatism drugs with heart failure (this caused the product to be wrongly taken off the shelves for several months) to allowing fake articles (such as one written by Michael Eisen saying he had discovered a new bacteria which uses arsenic in its DNA instead of phosphorus) to be published in highly regarded journals.

Evidently, drastic changes need to be made to fix the problems with the peer review system. Forum users and bloggers have taken to the internet to suggest their own alterations to the system. One such user, under the username ‘Darren Baker’, wrote on the famous ‘New Scientist’ blog that; “I would suggest a crowd sourced peer review website, much like Wikipedia, with the members reviews given weight based on their areas of expertise, and history of peer-reviewing. That way there could be a central location for media outlets and private citizens to check on the strength of claims.” Other users on the same blog have suggested things such as allowing videos of experiments to be posted (so that easier analysis of methodology would be possible) and allowing scientists to appeal a rejected paper if he or she believes that a mistake has been made in the peer review process (this is particularly significant; often if a paper is not published, it will be retracted and forgotten about).

With the boundaries of science being pushed further than ever, it is more important in the present day to ensure the validity of our methods and findings. Scientists around the world need to be able to rely on the findings published in scientific journals, so to design better experiments and theories for the future. The peer-review method, whilst currently flawed in many ways, is an instrumental part of removing bogus research and furthering pure scientific endeavour.

Dylan Lynch

Dylan is an SF Medicinal Chemist studying at Trinity College Dublin, and is the Science & Technology Editor.