More Options

Vaccines, Autism, and the Promotion of Irrelevant Research: A Science-Pseudoscience Analysis

Feature

Craig A. Foster and Sarenna M. Ortiz

Skeptical Inquirer Volume 41.3, May/June 2017

Larry Kusche’s review of the Bermuda Triangle mystery (1986; 2015) provided one of the clearest victories for reason over rumor. His method remains convincing because it was so straightforward. Kusche demonstrated that many disappearances had been wrongly attributed to the Bermuda Triangle; the actual events either did not happen as reported or likely occurred outside of the infamous area. Kusche also demonstrated that the rate of actual disappearances within the triangle did not seem to differ meaningfully from the rate of disappearances in other parts of the ocean. In so doing, Kusche highlighted two issues that occur frequently in pseudoscience. The first involves reporting events inaccurately, and the second involves relying on handpicked observations rather than a representative set of observations (Hansson 2013).

Vaccinations and Autism: Inaccuracies and Anecdotes

Much of the debate surrounding vaccines and autism has been similarly based on straightforward considerations about whether the reported evidence is accurate and whether a representative set of evidence supports the vaccines-cause-autism claim. This debate was triggered by Andrew Wakefield and others’ (1998—Retracted) research involving twelve children who were referred to a pediatric gastroenterology unit. Wakefield et al. reported that all twelve children had experienced developmental problems at varying intervals after exposure to the measles, mumps, and rubella (MMR) vaccine; nine of these developmental disorders were identified as autism with a tenth being questionably identified as autism. Wakefield et al. used these results to suggest that the MMR vaccine could contribute to a syndrome involving gastrointestinal problems and regressive autism. This research fueled the well-known concern about childhood vaccination and its alleged contribution to autism—a genie that shows no sign of returning to its bottle.

Even if the Wakefield et al. results had been legitimate, the small sample size and selective nature of the sample should have encouraged a cautious interpretation. Nevertheless, the evidence was at least broadly interpretable: a small group of children received the MMR vaccine and subsequently experienced developmental problems. Of course, this evidence was not accurate. Wakefield created fraudulent results presumably for financial reasons (Deer 2011). Subsequent research revealed no evidence that childhood vaccine administration elevated the rate of developing autism (Taylor et al. 2014). In the absence of any scientific connection between vaccinations and autism, promoters of the vaccines-autism link could still handpick observations. Anti-vaccination proponents pointed to the many vaccinated children who later developed autism (noted by The Logic of Science 2016). The best-known example has probably been actress Jenny McCarthy’s son. In a strange twist, some have raised the possibility that McCarthy’s son did not actually develop autism (Rubin 2008).

We view the handpicking of confirmatory observations with understanding. It is natural for humans to seek explanations for difficult events, and the temporal proximity between a vaccination and the diagnosis of autism could certainly feel causal. Nevertheless, this aggregation of anecdotes, no matter how broad, is not science. Most objective observers can understand that if millions of children are vaccinated and a fraction of children develop autism, then many children who have been vaccinated will subsequently develop autism, even in the complete absence of any cause-effect link. Obviously, many people still believe that vaccinations cause autism, but at least the science and pseudoscience surrounding this debate remains relatively clear. The scientifically minded can point to the absence of any correlation between childhood vaccination and the development of autism. Proponents of the vaccines-autism link can point to the number of children who were vaccinated and later developed autism.

Vaccines and Autism: The Promotion of Irrelevant Research

This context is important for understanding an interesting tactic that subsequently developed regarding vaccinations and autism. The supposed science behind the vaccine-autism link came back. The vaccines-cause-autism community began offering several non-Wakefield studies as evidence of supportive science. As far as we can tell, this tactic was popularized by blogger Ginger Taylor in 2007 when she published a list of “just over a dozen studies” supporting the link between vaccines and autism (see Taylor 2013). This list has since expanded to well over 100 studies. Taylor’s work is paralleled by other lists to include Walia’s (2013) list of twenty-two studies, Adl-Tabatabai’s (2015) list of thirty studies, and Anti-Vaccine Scientific Support Arsenal’s (2015) list of twenty-six studies, the latter containing mostly perspectives and reviews. To illustrate the dissemination of this argument, the Activist Post’s Facebook page, where Walia’s list was posted on August 24, 2015, appears to have more than 500,000 followers presently. We do not need to address these studies specifically, as the collective implication of these studies has been debunked sufficiently by scientific research (e.g., see Taylor et al. 2014) and by scholars who have reviewed the concerns associated with many of these studies (e.g., Ditz 2013; The Logic of Science 2016).

Instead, our purpose is to highlight the promotion of irrelevant research as its own developing characteristic of pseudoscience. This tactic shares a connection with the old pseudoscientific tactic of using scientific-sounding language (Shermer 2002) because both tactics might make a community appear more scientific. Nevertheless, the promotion of irrelevant research goes much further by pointing to purportedly important scientific findings.

Radner and Radner (1982, 36) gave a nod toward the promotion of irrelevant research when they described the “grab-bag approach to evidence.” According to Radner and Radner, pseudoscience will use quantity of evidence (the grab bag) over quality of evidence in an attempt to wear down opponents. Radner and Radner’s grab bag focused mostly on the continued offering of confirming observations (e.g., Bermuda Triangle disappearances) or questionable pieces of evidence (e.g., an old jet-shaped figurine as evidence of ancient aliens). Radner and Radner also included the misuse of research findings in their grab bag after explaining that pseudoscientists are reluctant to “weed out” bad evidence from a scientific debate. To illustrate, they noted that parapsychologists continued to use the results of a flawed research design as evidence for psi.

At the same time, Radner and Radner did not really describe the promotion of irrelevant research as a specific method for making pseudoscience look like science. This tactic, as it has been used in the vaccines-autism domain, involves much more than the inclusion of a dubious study or two that lie at or near the center of a science-pseudoscience debate. The promotion of irrelevant research is an active aggregation of several questionable or peripherally related research studies in an attempt to justify the science underlying a questionable claim (see also Barrett [2008], who mentioned this tactic briefly). It includes, among other things, (a) results that have dubious legitimacy, (b) results that possibly occurred due to chance, (c) results based on inappropriate statistical procedures that create a false perception of a relationship, (d) results coming from poorly controlled research, (e) results where the supposedly harmful aspects of vaccines were manipulated at much stronger levels than is actually present in a vaccine, (f) results where the dependent variable was tangentially related to autism but was not autism (e.g., gastrointestinal problems), and (g) results containing multiple explanations due to confounding variables (see Ditz 2013; The Logic of Science 2016).

The promotion of irrelevant research potentially changes the landscape of the overall discussion surrounding vaccines and their relation to autism. It can shift the argument away from the misuse of handpicked examples or a manageable number of fallible research designs (all the while ignoring the broader absence of any systematic link between vaccinations and autism). It can move the argument to an assortment of scientific findings with questionable relevance and legitimacy (all the while ignoring the broader absence of any systematic link between vaccinations and autism). By doing so, the vaccines-cause-autism community no longer ignores the entirety of the science regarding vaccines and autism. They have instead developed and promoted a scientific debate that does not actually exist in science. Instead, this bogus scientific debate takes place in an electronic world between people who are usually consumers of science rather than being scientists themselves. Among actual scientists, this does not seem to be an issue. Taylor et al. (2014), in their statistical integration of several studies that examined vaccination and autism development, referred to almost none of the studies provided in the lists of research supposedly supporting the vaccination-autism link. Other scientists who have conducted research involving vaccination and autism have demonstrated similar levels of disregard (e.g., Uno et al. 2012).

We can illustrate the promotion of irrelevant research by considering this development relative to other questionable health-related claims. A Google search for “studies that show that vaccines cause autism” revealed the four aforementioned lists within the first thirty-two non-advertisement results. A Google search for “studies that show that new age crystals” did not immediately reveal similar lists within the first thirty results. A Google search for “studies that show that chiropractic” did reveal lists of research supporting chiropractic medicine that appear similar to the lists used to promote the vaccines-autism link. Hall (2014) reviewed the top chiropractic studies of 2013 (Luck 2013) and offered several methodological concerns. There is, however, a critically important distinction between this debate and the bogus scientific debate in the vaccine-autism domain. Hall reviewed the top chiropractic studies as reported by a chiropractic website. This stands in obvious contrast to proponents of the vaccines-autism link pointing to a set of research findings that appear to be generally irrelevant to the actual scientific debate.

Why has the promotion of irrelevant research occurred so prominently in the vaccines-autism domain? The anti-vaccination movement boomed after the publication of what appeared to be legitimate scientific research (Wakefield et al. 1998—Retracted). This caused many individuals to become firmly entrenched in the vaccines-cause-autism view. This initial entrenchment was surely reinforced by not vaccinating children, encouraging others to avoid vaccinating children, or both. It is difficult to admit wrongdoing in this regard because the consequences involve the well-being of children. The debate is also interdependent because this issue involves the well-being of unvaccinated and vaccinated children. This interdependence likely gave vaccination supporters additional initiative to press the anti-vaccination community for scientific justifications.

The vaccines-autism link has also been more neatly debunked than other forms of health-based pseudoscience. The vaccines-autism link is more easily examined because the results are not complicated by psychological rather than physical explanations. For instance, chiropractic medicine is difficult to investigate because it is seemingly impossible to create a proper control condition where participants believe they received chiropractic treatment when they did not. The vaccines-autism debate is not complicated by a placebo effect in this way. In fact, the vaccines-cause-autism theory could subtly encourage autism spectrum disorder diagnoses in vaccinated children; of course, this pattern of results has not been revealed in extensive scientific examinations (e.g., Taylor et al. 2014).

We also believe that the development of the Internet and social media has enabled this pseudoscientific tactic (see, e.g., Kata 2012). This promotion of irrelevant research has occurred overwhelmingly—possibly exclusively—in this context. The Internet provides quick access to a wealth of scientific and pseudoscientific information that allows groups on both sides of this debate to disseminate information swiftly and widely. This seems to have enhanced the ability of the scientifically minded to press supporters of the vaccine-autism link for the science behind their claims. Similarly, the Internet surely enhanced the ability for supporters of the vaccine-autism link to find and list several research studies that look, at first glance, as if they provide a noteworthy scientific argument. Taylor’s (2013) original list was clearly intended to address concerns that there is no scientific evidence for the vaccine-autism link. If the Internet and social media hastened the promotion of irrelevant research, it explains why the Radner and Radner (1982) grab bag focused primarily on confirmatory observations and poor examples of evidence. It would have been difficult for nonscientists in 1982 to quickly piece together and share lists of any research that has some remote possibility of supporting a pseudoscientific claim. The continued ability for individuals to do so suggests that the promotion of irrelevant research is likely to continue as a pseudoscientific tactic.

Pyrrhic Victories and Practical Implications

Unfortunately, the promotion of irrelevant research is intentionally or unintentionally clever because it can obscure the distinction between science and pseudoscience. Science advocates can explain fairly easily why handpicked observations or particular research designs might be misleading. It is more difficult for science advocates to address an extensive list of research findings that have dubious relevance and legitimacy. Yet if these lists are not addressed, newcomers might initially see science that purportedly promotes both points of view, which can create a sense of false equivalence (Skeptical Raptor 2015).

At the same time, entering this debate could create a series of Pyrrhic victories. Scientifically savvy individuals who critique the promotion of irrelevant research might win several battles while experiencing a setback in their overall campaign—a campaign that is waged with arguments rather than soldiers in an effort to promote reason rather than empires. Those who truly love science will be tempted to enter this debate knowing that they ultimately have a winning hand, but the vaccines-cause-autism community can offer a long list of weak counterarguments in the form of fraudulent research, poor research, tangentially related research, and alleged pro-vaccination conspiracies. Sadly, by the time the arguments about the science underlying vaccinations and autism are hashed out, newcomers to this issue might be understandably fatigued and confused about what they should believe (Radner and Radner 1982; Skeptical Raptor 2015). This can force science to rely on authority rather than a digestible explanation of the existing scientific evidence. Ironically, the reliance on authority to confirm belief is another tactic commonly associated with pseudoscience (e.g., Hansson 2013).

It might be wise for those who wish to promote science and reason to steer clear of the specifics associated with handpicked irrelevant research. Getting into the weeds of this debate might encourage people to believe that all scientific research endeavors are equally compelling and that the group with the greater number of supposedly supportive studies is the winner. It is probably more effective to keep the focus on the basic scientific principles that are more easily understood. If vaccines cause autism, then vaccinated children should be developing autism at an elevated rate compared to non-vaccinated children. There is no evidence of this pattern (see Taylor et al. 2014 for an introduction). The tangential findings provided by several questionable and legitimate studies consequently lack any substantial relevance. Only those who are really interested in the details of these generally irrelevant studies should bother with them. They can be directed to thoughtful reviews (e.g., Ditz 2013; The Logic of Science 2016).

Perhaps more important, the promotion of irrelevant research is, oddly enough, an acknowledgement that science matters. If science did not matter, there would be no reason to offer lists of supposedly supportive scientific studies. This creates two contradictory stances that the vaccines-cause-autism community can be asked to clarify. First, should decisions about vaccines and autism be based on the scientific evidence? It seems that some in this community believe that science matters (e.g., Taylor 2013) whereas others question the utility of science (e.g., Jameson 2015). Second, is there a conspiracy that stifles research supportive of a vaccine-autism link (e.g., Olmsted 2014)? If so, why are there so many published studies that supposedly support that link?

The answers to these questions need to be established before, not after, a debate ensues about whether science supports the vaccines-autism link. Clarifying the ground rules is clearly necessary because the vaccines-cause-autism community can claim that their science is legitimate but that any science discrediting the vaccine-autism link is biased by some type of nefarious pro-vaccination agenda (see, e.g., Jameson 2015). This community can also argue that there would be additional research supportive of the vaccine-autism link if it had not been suppressed by a nefarious pro-vaccination agenda (see, e.g., Olmsted 2014; Mikkelson 2015). If the vaccines-cause-autism community wants to have a legitimate scientific debate, we are confident that members of the scientific community would be happy to entertain it. It just needs to come with a commitment that scientific findings cannot be omitted for undocumented impropriety and that suppositious research findings that have been suppressed by some unsubstantiated pro-vaccination agenda would not be considered.

Conclusion

The promotion of irrelevant research reveals a fundamental contradiction. It acknowledges the importance of science but disregards the most informative scientific studies and the general consensus of the scientific community. Our science versus pseudoscience analysis of this development serves two purposes. First, we hope that it encourages an effective response to the promotion of irrelevant research in the vaccines-autism domain. Second, we want to highlight the promotion of irrelevant research as an important pseudoscientific tactic in its own right. We believe that the promotion of irrelevant research will expand as a pseudoscientific tactic, and promoters of science and reason should therefore be prepared to identify and address it.



(The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the United States Air Force Academy, the Air Force, the Department of Defense, or the U.S. Government.)

References

Craig A. Foster and Sarenna M. Ortiz

Craig A. Foster is a professor in the Department of Behavioral Sciences and Leadership at the United States Air Force Academy. He received his doctoral degree in social psychology from the University of North Carolina at Chapel Hill. He currently serves as the course director for a year-long statistics and research methods sequence. He also conducts research in the areas of scientific reasoning and pseudoscience.

Second Lieutenant Sarenna M. Ortiz is an admissions advisor at the U.S. Air Force Academy. She is a graduate of the U.S. Air Force Academy, where she majored in Legal Studies and Behavioral Sciences and Leadership.