Many randomized controlled trials (RCTs) comparing nutrient interventions with placebo/control have failed to replicate the (usually protective) associations between nutrients and risk of chronic disease found in large-scale observational data.^1–5 High-profile examples of such RCTs found no evidence of intervention effects for fiber and colon cancer,⁶ vitamin E and cardiovascular disease,⁷ vitamin E and lung cancer,⁸ β-carotene and coronary events,⁹ vitamin C and cardiovascular disease,¹⁰ or folate and cardiovascular disease¹¹, and even identified adverse effects of nutrient supplementation (e.g., β-carotene and cancer⁸). The discrepancies between the findings of RCTs and epidemiological studies raised serious questions about the currently used approach for determining whether the evidence base is adequate to justify launching a large-scale RCT, with hard endpoints as the outcome measure.

Deciding which specific nutrient-outcome association to further evaluate in human intervention trials is challenging. Simply having a plausible epidemiological support is probably not enough; all aforementioned negative large RCTs were motivated by hypotheses vetted in epidemiological data or in smaller RCTs examining surrogate outcomes. In previous work, we hypothesized that additional critical components pertain to the maturity and reliability of the relevant evidence base — that is, the strength of the data supporting a potential nutrient-disease association, the biological plausibility of the association, the reliability of existing data, and the likelihood of bias and systematic errors affecting interpretation of the available data. The evidence base is formed by the interplay of various translational paths, in which an initial hypothesis-forming observation supports subsequent research and is eventually “translated” to interventions for preventing or treating human disease.

Understanding the translational paths that shape the evidence base may result in insights on why epidemiological and randomized data disagree or agree. To some extent, the translational paths can be assessed with citation analysis, which is a qualitative and quantitative representation of citations between publications. In previous work, we conjectured that nutrient associations where RCT and observational data is concordant have a more extensive and mature evidence base compared with associations where the data is discordant.¹² Otherwise put, differences in the observed flow of information (as captured by citations that are received or made between publications) through the various translational paths and the content of the propagated information are associated with concordance or discordance in the results of observational studies and RCT. For example, a limited evidence base and information flow may indicate inconsistency of study results and thus may be associated with topics where RCT and observational studies disagree. Reciprocally, a large evidence base with higher information flow may indicate consistency of findings and general agreement between RCT and observational studies. Of course, these are not one-to-one relationships; it is conceivable that profound inconsistencies and disagreements between studies could lead to considerable discourse among investigators, which in turn would increase information flow.

In our previous paper, we empirically explored this hypothesis by analyzing and comparing characteristics of the citation networks in two nutritional associations of disease, one where the two research designs generally agree (polyunsaturated fatty acids and cardiovascular mortality), and one where they disagree (vitamin E and cardiovascular mortality). We performed systematic reviews in each example and constructed and analyzed the respective citation networks. We identified no differences between the characteristics of the two networks. Most interestingly, we found no evidence that observational studies predated RCTs in the translational process in either example.¹² Here we expand our previous work to include a large systematic sample of associations between nutrients and health outcomes that were examined in both epidemiological studies and in RCTs. We construct and analyze citation networks to describe translational paths, and assess the relationship between metrics that describe the citation networks and the concordance of the summary findings across the two research designs.