Selective citation and the shaping of scientific knowledge: citation network analysis and the diet-heart debate

Abstract

Scientific knowledge is based, in part, on empirical evidence. Scientists contribute to a particular evidence-base by publishing the results of their experiments and observations, but in writing their papers they also use evidence produced by others. Scientists, however, are often selective in the evidence they use and can differ in their interpretations of that evidence. But how selective are scientists when evaluating specific scientific claims in the literature? And how does the use and interpretation of past studies shape what is eventually taken to be scientific knowledge? This thesis develops an approach based on variants of Citation Network Analysis to study practices of selective citation and their consequences. I develop this approach around a specific case study that traces the emergence and evolution of the conjecture that dietary fats are causally related to the development of coronary heart disease (CHD), and the controversies that this provoked, in the scientific literature between 1900 and 1984. First, I develop a novel method to detect citation bias, the act of selectively citing only evidence that supports a particular conclusion, by introducing sub-graph analysis to claim-specific citation network analysis. I apply this to trace how the results of the first four clinical trials examining the effectiveness of dietary fat modification in the secondary prevention of CHD were cited by reviews of literature prior to a major consensus conference in 1984. By this, I demonstrate that 82% of reviews that reached supportive conclusions regarding the effectiveness of dietary treatment cited only evidence that supported that conclusion. Second, I adapt the claim-specific method to examine how the results of the first prospective cohort study to return findings related to the relationship between dietary fat and CHD were cited in the literature over a 7-year period. This study reported no association between dietary fat and CHD, but also reported on a plethora of other factors. Its findings were cited in many different ways that appeared to reflect the research interests of particular communities. I demonstrate that the dietary fat findings made the biggest impact not on those interested in the relationship between dietary fat and CHD, but in a community advocating a link between sugar consumption and CHD. Third, I develop a method for capturing a large body of literature representative of the ‘paradigm’ in which the diet–heart link arose and was evaluated via a layered Boolean search query methodology. From these retrieved data, I construct a large citation network that reconstructs the state of the literature prior to 1985, demonstrate network analytic methods for its validation, and construct a citation map of this research area that makes use of modularity analysis for community detection and temporal analysis. By this, I demonstrate that time played a major role in structuring the network – authors tended to cite recent literature. Authors also tended to cite other papers that shared similar research questions, and this appeared to cluster documents into sub-communities based on research focus. These results suggest the presence of a dynamic research front that, over time, focussed on different questions regarding the validity of the diet–heart hypothesis. Finally, I apply Main Path Analysis to this network to understand how the link between dietary fat and CHD emerged and progressed through the literature. Main Path Analysis detects citation-paths that are most used in a body of literature – a method to understand how selective citation is involved in the development of the main research stream over time. I interpret the results by the reading of the identified documents and demonstrate that highly cited documents tended to be those that preserved the hypothesis rather than those that posed inconveniences to it. The four major analyses performed throughout this thesis suggest that selective citation played an important role in the development of the diet–heart link. While I applied this approach to a specific case-study, its basic logic and methods ought to be applicable to other cases.