When and How to Review a Paper. Part II

Congratulations! Imagine:- you have been asked to review a paper and, having considered the issues raised in Part I of this blog, you have decided that you can, and do want to review the manuscript. But how do you do it?
Below are the 7 steps that will help you through the process:
Step 1: It’s a good idea to give yourself time over a few days for the review process. For instance, you could read the full paper first and then leave it for a day to mull over the arguments and the logic of the broad conclusion. You may find that something does not feel right but you need a little time to work out what the issue is. This can be harder to do if you have only given yourself a constraint time space to read and critique the paper as well as write a considered review in one go.
Step 2: Many journals will give you an outline of what they want you to judge initially, in the form of a kind of a ‘multiple choice’ tick process (is it suitable for the journal?, is it new information?, etc.), followed by a free-form area that will be shown to the author(s) of the manuscript. Most of these tick boxes you can complete after your first read of the manuscript but sometimes it’s best to wait until you have worked through your review in detail and come back to this section.
Step 3: Your free-form assessment, which will be given to the author(s), is the nub of your review. You traditionally start with a short paragraph, summarising the manuscript in 3 or 4 sentences. This is done for the benefit of the editor who is most likely not an expert in the field, but it’s also a really good way to clarify in your own mind the core of what the manuscript is about.
Step 4: Most reviews I’ve seen separate criticisms into major issues (in the order they appear in the manuscript) of experimental design or logic; and then a list of minor issues including small things you picked up such as errors in figure legends (use line or paragraph numbers). This also allows you to make general comments on the novelty of the work (backed up by evidence) as well as tackle issues that may impact on the manuscript as a whole (such as an error in logic that affects the methods and the results).
Issues of grammar and spelling are the editor’s job, but, it is a good idea to point out any spelling errors that require content knowledge (such as names specific to the field). If grammar or spelling is a major issue, do make a more general comment about this, especially if it impacts on how efficiently the science is communicated.
Follow this link for some advice from several journal editors on what they are looking for in a review. It’s always a good idea to check the reviewer guidelines of the specific journal you are reviewing for, to get more detailed information of what they are after.
There is no question that critiquing and evaluating someone else’s work can be difficult and time-consuming. However, time spent on this will train and improve your own critical abilities, knowledge of your field and approach to your own research, which is invaluable.
Step 4: If you find evidence of fraud, plagiarism or other scientific misconduct, contact the editor immediately after you have collected evidence to substantiate your claim, and discuss further action with him or her.
Step 5: There is usually a final section where you can leave a comment that only the editor will see. You use this section for confidential information or if you want to explain in more detail why you have significant reservations about the manuscript, but are still keen for the manuscript to be published – or explain why it should be rejected. Under no circumstances be disrespectful to the authors here just because they will not see what you write. Your professional conduct is under review by the editor so personal attacks will certainly weaken the impact of your review, as well as damage the good impression you may have made with your thoughtful review in the section above.
Step 6: Once you finished your review, it’s a good idea to leave it over night and send it the next day after having re-read it. Have you outlined your criticisms objectively and logically? Is your tone respectful and factual? Have you missed anything? Does your criticism and level of concern about any aspects of the paper match with your suggestion to the editor of how to proceed with this manuscript (such as outright rejection, resubmit with revision, accept without revision)? If everything still stacks up the next morning, you are good to go: a job well done!
Many journals will automatically send you their decision letter together with the other reviewers’ comments once the fate of the manuscript is decided. This is a great way to learn from other reviewers and improve your judgment. If you haven’t received any follow-up it is perfectly fine to contact the editor and ask for feedback on your review.
Useful resources:
 – Elsevier is publishing a quarterly newsletter for reviewers with useful information. One of the articles for instance is a handy summary of the most common statistical errors found in scientific papers.
– You can find a good outline of issues around peer review in Nature’s peer review debate.
– BMJ (British Medical Journal) provides quite extensive reviewer training materials including a good power point presentation (under Objective 2) of what editors want from the peer review, as well as more extensive training material, including a review exercise with comments (Objectives 3 and 4). This resource is focussed on medical science publications.
Do you have questions or did I miss any major aspect of writing a review? What are your experiences with reviewing? I’d love to hear from you so please leave a comment.

Thanks, Kerstin

'7 Keys for Postdocs: The Essentials for
Taking Control of Your Future'

Free Report and Self Assessment

, ,

One Response to When and How to Review a Paper. Part II

  1. Sara Eastwood May 28, 2012 at 9:45 pm #

    A great article, outlining an excellent methodology that can be applied to all types of reports and theses, as well as papers, thanks!

Leave a Reply