Publish and Perish, part 21: Citation Analysis

[Context: read the previous part or start at the beginning. TW: bad amateur fiction!]

Back at home that evening, Gerald thought of his ghost-town lab tour earlier in the day and considered the value of his scientific output to date.

To his colleagues, and according to his CV, he had a strong track record. He had gotten some small-to-medium-size grants and subcontracts here and there, and when encouraged or forced to give a departmental seminar he always seemed to have new data and new ideas. Above all, he had made it into some decent peer-reviewed journals with papers that were cited by similar papers in similar journals. At a small college like his, that was about the best that you could hope for: supervise undergraduate projects, publish the ones that worked out well, and get cited for those successes.

But were his publications actually advancing the field of biology? Or were they just demonstrations of competence without any particular real-world value?

Gerald had not previously given much attention to how his papers were being cited in the literature. He decided to start with Chernoff & Cutler, the paper based on Lois’ undergraduate thesis. It was a nice little study — an early sign of Lois’ strong scientific instincts — showing that, when their enzyme of interest was given a hexahistidine tag (to make purification easier) at either the N-terminus or the C-terminus, the enzyme kinetics were almost identical, suggesting that the tag did not greatly affect the enzyme’s 3D structure or function.

Since its publication in the Journal of Biological Chemistry a decade ago, C&C had been cited 51 times, a respectable total that made this paper one of his more “popular” ones outside of his graduate and postdoctoral work. But why was it being cited? With Google Scholar as his starting point, Gerald tried to access each of the 51 articles.

The first article could not be accessed online.

The second article cited C&C in support of the general point that the enzyme’s pathway was of interest as a potential drug target. A true but obvious point also made by dozens of other papers.

The third article incorrectly referred to C&C as characterizing an enzyme that they hadn’t actually studied.

The fourth article cited C&C as using a certain wash buffer — not a buffer they had invented or tested against alternatives, just one they had used!

The fifth article mentioned C&C’s Km value for its enzyme’s substrate, then noted with apparent satisfaction that this was “within two orders of magnitude” of the present study’s value.

The sixth and seventh articles used C&C more substantively because they were subsequent papers also coauthored by Gerald.

The eighth article included C&C in its reference list but didn’t refer to the paper in the main text.

After a few more depressing rounds of this, Gerald switched to a more recent paper of his, Ocampo et al., where they had coined a phrase (“druggability paradox”) that Gerald had hoped would catch on as a concept. Had others picked up on that idea and cited them for it? Alas, they had not.

The best that could be said was that other researchers were aware of and acknowledged these papers’ existence. That aside, the citations were tangential to the new studies; the citations’ main purpose seemed to be fulfillment of expectations that the Introduction and Discussion sections be stuffed full of references to somewhat related research. Gerald had to admit that he often cited other papers in this superficial way. He had hoped that others’ usage of his papers would be more meaningful, but this did not appear to be the case.

[Update: the story continues with part 22.]