We analyzed the citation networks of publications on the associations of n-3 PUFA and vitamin E with cardiovascular mortality. We observed that, in both examples, citation networks were grossly similar with respect to their quantitative characteristic (such as the number of articles and citation relationships among articles and the connectivity of the articles). This was also true for citation networks limited to the subset of articles that described primary data in humans and were pertinent to the association between the nutrients and clinical cardiovascular outcomes. In both examples, there was no indication that the research that led to the index RCTs and observational studies proceeded in a sequential fashion, with observational studies preceding RCTs. For both examples, at least two RCTs enrolled their first patient before or on the same calendar year that the earliest-published index observational study appeared in the literature.

Research builds on previous findings. Biomedical knowledge advances as scientific data and its interpretation are communicated towards informing further research. Scientific publications are the generally accepted vehicle for such communications, and the citation networks they form are a representation of the flow of biomedical knowledge through various translational paths ²⁷. From this point of view, analysis of citation networks may provide insights on the size and maturity of the relevant evidence base. Further, in some extreme cases it may demonstrate the propagation of erroneous conclusions, and help understand the formation and establishment of unfounded belief systems in the medical community⁸².

We hypothesized that nutrient associations of disease where RCTs and observational studies agree in the direction and significance of their findings may differ in the size of their evidence base, and the patterns of information flow in it. While we found small differences in the quantitative characteristics of the citation networks of the n-3 PUFA and the vitamin E examples, these were not dramatic. Thus, there is no suggestion that quantitative analyses of citation relationships can distinguish topics where RCTs and observational studies agree or disagree. Our negative finding is easy to explain. Only the most dramatic aberrations in a translational path would have been identified in an analysis of citation relationships. An example would be a whole body of literature generated from an unwarranted extrapolation of previous findings. There is no evidence that any such aberrations occurred in the two examples we explored.

Notwithstanding the aforementioned observations, this paper introduced methodologies for the quantitative analysis of citation networks as proxies for understanding translational paths. In analyzing the two examples, we found no empirical support of the notion that research translation progresses in a linear fashion, at least in the rightmost end of the translational spectrum, where hypotheses are being evaluated in humans. If anything, the index RCTs and observational studies (the reports that represent the state of knowledge in the two examples) were published over the same time period. If anything, in Figure 6 question whether the design and launching of most index RCTs could have been informed by the majority of index observational studies.

There are limitations to using citation analysis to understand translational paths in a given topic. Citing previous research is a complex process. As much as we would like citation practices to be impartial and scientific, they are influenced by personal beliefs, biases, and preferences, and are subject to citation distortion. The latter includes citation bias, i.e., when one systematically ignores articles that contain content at odds with ones claims; citation amplification where a lot of citations propagate a belief without any evidence support; and citation invention, which includes citing content but ascribing different meaning to it and converting an hypothesis to fact through citation alone ⁸². Our searches may have missed early publications that potentially influenced subsequent work, either observational or experimental, and the majority of the publications in the citation network (with the exception of those included in the systematic reviews), were not assessed in detail for content, pertinence to the association, or methodological quality. The two examples chosen both had cardiovascular mortality as an outcome and results might have been different for nonmortality outcomes. Publication bias, which occurs as a result of researchers publishing only studies with significant positive results and ignoring studies that report nonsignificant results, also affects the citation network. Lastly, much of the work cited for the two examples was published before a registry of clinical trials was established (www.clinicaltrials.gov). Hence, we cannot rule out the possibility that reporting bias confounded our conclusions.

Despite the above limitations, we believe that citation analysis is one of the few representations of the translational process that can be objectively quantified. It merits attention as it can provide a framework to analyze the series of research steps that led to RCTs in humans, and perhaps identify unwarranted extrapolations in the translational process, if any exist⁸². We believe that an empirical exploration of citation relationships for a larger number of nutrient/disease associations is necessary to adjudicate our conclusion that most index RCTs had likely not been informed by the mass of corresponding observational studies in either example examined in the current study.