Civil discourseDecember 30, 2013
A recent, typically excellent post at Dynamic Ecology addressed the question of “How do you critique the published literature without looking like a jerk?”
While I like Brian and Jeremy’s suggestions, they don’t capture the extent to which perceptions of jerkiness depend on very specific choices in wording, rhetorical structure, etc. I want to emphasize here that small changes can make a big difference in how you are perceived.
Brian and Jeremy did provide the following language/writing tips (paraphrased by me):
1. Don’t make ad hominem attacks.
2. When possible, identify possible solutions to the problems you’ve identified.
3. Don’t heap too much scorn upon one individual paper or scientist if the problem is common to multiple sources.
4. Focus on facts rather than opinions.
To these, I add the following additional pointers for avoiding the label of “jerk.” (Some were taken from A guide for new referees in theoretical computer science by Ian Parberry, which I consider relevant because standards for pre- and post-publication review should be similar.)
5. Define the viewpoint from which you conducted your evaluation. What do you know about and care about? What do you NOT know about or care about? Confess possible biases.
6. Acknowledge the positive aspects of what was done.
7. Be as specific as possible in your criticisms. Statements like “the data in Figure 5 were misinterpreted” are both more justifiable and less catty than “this study adds nothing to the field.” If you are questioning one particular paper, consider your target to be the paper rather than the scientist(s) who wrote it. “What’s wrong with this paper?” is usually a reasonable question to ask, but “what’s wrong with these scientists?” often registers on the jerk-o-meter. Also be specific in providing references. Claiming support from unspecified sources is sloppy and rude, but even incomplete citations such as “Johnson 2012” may be more ambiguous than helpful.
8. Give the most space to the most important problems. Don’t dwell on minor flaws. Harping on spelling errors makes you seem like a jerk.
9. Avoid unnecessarily dramatic language. In a manuscript review, I once identified four problems as “critical flaws.” A colleague noted that my concerns would be just as clear if I used a softer phrase such as “main flaws.”
In compiling this advice, I became curious as to how well I follow it, so I rated my past critiques of Lore of Running by Timothy Noakes, Born to Run (part 1; part 2) by Christopher McDougall, “Metabolic Factors Limiting Performance in Marathon Runners” by Benjamin Rapoport, Wheat Belly by William Davis, and “Misconceptions Are So Yesterday!” by April Maskiewicz and Jennifer Lineback. Results are below.
By my own reckoning, I’m not a complete hypocrite, but there certainly is room for improvement.