The time-honored tradition of writing a response (or rebuttal) to a published article is vital to the progression of science, providing active debate, pointing out flaws in papers, and ensuring the validity of claims (read more here). But are we always doing it right? And with the correct intentions?
Associate Professor Trevor A. Branch from Washington University recently posted a thread about this on Twitter (see it HERE), which I considered particularly timely as I was in the process of preparing a reply, to a response, to an article my supervisor and I had published.
Trevor’s key points were that we shouldn’t be writing responses as a way of making publication lists look numerically more impressive, and there are simple approaches that can be taken to ensure that the response is needed in the first place (i.e. are you sure you have a meaningful correction to make or have you just misunderstood something).
You can read our original paper HERE, the response letter HERE, and our reply to that HERE. The short summary of this is (1) we wrote a concept paper that detailed a unique invasion phenomenon and defined it by the term currently being used haphazardly and without definition to describe that and other dissimilar phenomena. (2) the response letter had no problem with our highlight and discussion of the invasion phenomena, but took issue with the term we used, suggesting we had redefined something, and so they were now re-redefining it. (3) we then systematically responded to their claims, again making the point that we hadn’t redefined anything because the term was undefined, and so we had actually un-undefined it.
Fun play-on-words aside, it felt like I was just writing a response letter to a Reviewer that had only fleetingly sort to understand the article. A lot of “The authors claim we didn’t consider X, when actually it was a major component of our idea with a dedicated section and 2 Figures in the original article” ….. yep.
Let’s step through Trevor’s 6-step guide to best practice responding and see how we faired shall we:
1. Email the authors expressing your concerns. You might have misunderstood the paper or have it wrong.
I was first contacted by the journal editor with a copy of an accepted response letter to my article with an invitation to prepare a reply. Never received any direct contact from the authors of the response regarding our article at any point. Upon first read of their response, it was clear that most of the criticisms they had of our article were a misinterpretation of some of our points, that could have easily been corrected by either a more careful reading of our article, or by just shooting off an email.
2. If you found something wrong, and the original authors agree, consider writing a joint paper with the original authors that corrects the scientific record. This is much more powerful than a we-said-they-said narrative, which seldom changes people’s minds
This is an interesting idea from Trevor, and I would be interested to know how frequently this occurs. A key point I take away from this is about opening a dialogue to make sure that what you think is something wrong, is something that actually is wrong. Rather than a we-said-they-said narrative, the article-response-reply chain that I’m now a part of is really a we-said-they-claim-we-said-so-then-they-said-but-no-we-actually-said narrative…….which, yeah, is not at all useful. Just read our original paper and make up your own mind.
3. If you can’t reach agreement with the original authors, write your rebuttal, and share it with the original authors so they have time to write a response that can be published at the same time as yours. This is common courtesy.
We were invited to prepare a reply and given plenty of time to do so (and I would hope that would be the standard). But I guess there was also the option of dismissing the response as being inaccurate and not a meaningful contribution to the literature (which is essentially what our reply letter is). Which brings up a great point…..Was this letter even peer-reviewed?!? I wouldn’t have thought so given so much of it is simply mischaracterizing our original paper (which I would think would be cause for rejection). My supervisor and I prepared a 5000-word concept paper full of case studies and detailed discussion that was updated and improved by the considered comments and suggestions of reviewers. But now there is a 700-word letter designed to supersede our article (through reframing and repackaging our ideas with new terms) but without the same detail or critical oversight. Surely this is not good science. I would suggest another peer-reviewed 5000-word concept paper full of case studies and detailed discussion would be a better response ……..
4. Remember that the point of a rebuttal is not to get yourself a published paper. It is to correct the scientific record, so that flawed science does not persist.
Is the written response just about showboating or is there genuine and important criticisms? I propose a pretty simple test for determining this before you even read the content of a response….. Does the title of the response article include explicit reference to the article they are responding to?? Is it called “Something something: a response to Jones et al.” or is it just called “Something something”. I contend that if you do not reference the article in the title of your response you are being deliberately misleading by making it appear like your response is actually an original article. Did the authors of the response letter include reference to our original article in their title…….? NOPE. So we did the same thing with the title of our reply so that the chain didn’t look like their paper was the original article and we were the responders. So we were misleading in response to other misleading activity…… Do two wrongs make a right?
5. Remember that rebuttals seldom change people’s minds, get cited 1/10th as much as the original papers, and generally don’t have much effect on citations or perceptions of the original paper.
I don’t think there is any reason to ever cite the response letter or our reply letter. Unless you are looking for an example where somehow a response letter has been published that presents a key component of an original article, as an independent argument against the original article……… Our original paper on the other hand – definitely worth citing – whether you agree with the ideas or not (that’s the point of an ideas paper I thought!)
6. With these guidelines, you can at least reduce the chance of making enemies by writing rebuttals, increase the chance of changing the minds of the original authors, and sway the minds of the bystanders watching the debate. A joint correction is great science and politics.
Have the authors of the response letter ever gotten in contact with me? Nope. Have I ever gotten in contact with them? Nope (why should I make the first move……?). Are we enemies? I hope not, I find many aspects of their research really useful and I cite them regularly. Do I care? Care enough to dedicate an hour to prepare this I guess.
Mostly I’m just annoyed.