Peer Review

by Eva Amsen

Yesterday Nature launched a debate on peer review, which is open for everyone’s reactions on the accompanying blog.

There are currently five articles on the web focus site, each highlighting different problems in, and solutions for, the way scientific articles are currently published.
After reading all five, I have to agree with Joan E. Sieber, who ends her long list of problems in peer review with the remark: “One suspects that peer review is a bit like democracy – a bad system but the best one possible.”. I would like to add “…at the moment” to the end of it, though.

There are alternatives to peer review available already, such as Biology Direct (discussed in the debate). Biology Direct publishes articles with reviewer names and comments, and even publishes articles with very negative reviews. This leaves it up to the reader to decide whether or not this paper is scientifically solid, rather than to a few (anonymous) reviewers. However, it comes with the added risk of reviewers being too nice, because they don’t want their name associated with all too negative comments.

John Moore’s submission points out an often overlooked problem of the current peer review process: it’s not understood by the public. He points out that even science writers and journalists tend to assume that peer reviewed papers are by definition true. Of course this is not a golden rule: research advances and older findings may later be contradicted or shown to be not universally true (only in a certain system). An example he gives is the ridiculous myth that HIV doesn’t cause AIDS or doesn’t exist. I had heard this myth before, and I don’t even find it funny, just frustrating that there are people who believe this. In attempts to support this crazy claim, these people do try to refer to peer reviewed papers, but old papers, that are contradicted by many recent ones (which of course remain unmentioned…)
Compare this to something I mentioned in my review of An Inconvenient Truth: Al Gore compared all publications on global warming and found none to contradict each other. This is how to handle any publications, peer review or not. Imagine if in doing this he found one contradictory paper. That would still have supported his claim without any doubt, but it would have given an opponent one paper to wave around to proclaim the opposite. Clearly, the opponent would have been wrong, but this is exactly how the HIV-does-not-cause-AIDS people handle their reviewing.

On a lighter note, another problem with the peer review process:

(Click for the original comic if it’s illegible)

[Peer Review Debate links by way of Maxine Clarke, comic from PhD comics]

Save

Related Articles

2 comments

Maxine June 7, 2006 - 8:06 AM

Thank you for the very nice posting, Eva. Like the comic!
At Connotea we are collecting all the blog and other comments on this Nature initiative so I will add in a link to your post.
See .
Thanks again; we are looking forward to comments on our peer review blog.
all the best
Maxine.

Shelley June 9, 2006 - 7:17 PM

I read about the peer review thing at work yesterday (after seeing it in here). I’m looking forward to how it pans out.

Comments are closed.