Read on Twitter
For @chrishorn. How I review an article.
1. Read it front to back (skim methods). Review the citations in the intro. If you don’t know the article, pull them and make sure they support the authors points.
1. Read it front to back (skim methods). Review the citations in the intro. If you don’t know the article, pull them and make sure they support the authors points.
For key issues it’s worth making sure that the facts from references are really in there. Read the discussion. When you are done, write down the authors conclusions. Your job is to make sure that the conclusions are supported by the data.
2. Go through the results and methods together. Look at every figure, table and supplement. Write down what you conclude from the data and reconcile that with the authors results section. Be mindful of stats, and careful to read what each figure really is.
Is it the mean+\\-stdev? Is it median+\\- IQR? What are the units? Does it make sense? Given the methods, could you redo the experiment? Look for obvious methodological issues. If you are reviewing an article you need to be adequately familiar with the methods used.
3. Now I ask myself: A. Are the results valid (are their adequate controls, appropriate comparisons, stats, methods). B. What do *I* conclude from the data, and is that congruent with the authors conclusions. C. Are issues/discrepancies between my conclusions and the authors.
4. Ask if the results are novel. Do these data and conclusions advance the field.
5. Identify major criticisms: methodological flaws, lack of novelty, big issues. Tell the authors what they need to do to fix these issues. Add data, focus their conclusions to add novelty.
5. Identify major criticisms: methodological flaws, lack of novelty, big issues. Tell the authors what they need to do to fix these issues. Add data, focus their conclusions to add novelty.
Big things. These are “major revision” issues. Basically I don’t think the data support the conclusions or the results are not novel. But always tell the authors what they can do to fix it.
6. Identify minor scientific issues: over-reaching conclusions, minor stats issues.
6. Identify minor scientific issues: over-reaching conclusions, minor stats issues.
These are “minor revision” things that can usually be addressed with some change in analysis or added statement of “limitations.”
7. I don’t copy edit as a reviewer, but if the manuscript is poorly written (hard to read) or not clean (lots of typos/errors) I will note that to the authors and the editors.
8. A note about novelty: ultimately the editors will decide if they want to publish something, but as a reviewer it is my job to put the new article in the context of the literature.
If it just rehashes something well described, I need to see how if the article agrees or disagree with the lit and how much it adds to the field.
9. If I have big concerns (data really not supporting conclusion, references that don’t support the assertion in the paper, very poorly written) I will tell the editor to take a critical look at the paper.
I will also often pull up authors prior papers, especially if they are on a related topic and see how those fit in the mix.
