More Options

New Analyses Raise Doubts About Replicability of ESP Findings

Research Review

Scott O. Lilienfeld

Skeptical Inquirer Volume 23.6, November / December 1999

The 150-year history of research on extrasensory perception (ESP) has been plagued by what might be termed a consistent inconsistency. As University of Oregon psychologist Ray Hyman points out, this body of literature has followed an all-too-familiar pattern. Seemingly promising and potentially exciting effects using a novel experimental paradigm are reported, only to fizzle out upon closer scrutiny. Each round of replication failures engenders a brief period of disillusionment and disenchantment, which sets the stage for concerted attempts to find a new and improved paradigm.

Eventually, positive findings using yet another novel paradigm are reported, followed by another round of replication failures, and so on. Moreover, in contrast to the argot of what Imre Lakatos termed “progressive” scientific research programs, the lexicon of parapsychology is replete with terms describing the absence of effects. The “experimenter (shyness) effect” refers to the failure to obtain positive findings when skeptical researchers are present, the “decline effect” refers to the disappearance or marked diminution of ESP effects within a session following an initial run of positive results, and “psi missing” refers to ESP performance that is significantly worse than chance (see Gilovich, T., 1991, How We Know What Isn't So, New York: Free Press, for a good discussion). These terms underscore the absence of a crucial feature that is a hallmark of mature laboratory sciences, namely a readily transportable “experimental recipe” that can yield replicable results across independent laboratories.

This pessimistic state of affairs appeared to change, however, in 1994, when Cornell University psychologist Daryl Bem, in conjunction with the late University of Edinburgh parapsychologist Charles Honorton, published a remarkable article in Psychological Bulletin, one of psychology’s two most prestigious review journals. Bem and Honorton reported on a series of eleven studies using the "Ganzfeld” (a German word meaning “whole field”) paradigm, a method that originated in the 1930s. Subjects ("percipients”) in a Ganzfeld experiment are immersed in a uniform sensory field, typically by covering their eyes with Ping-Pong ball halves, directing a red floodlight toward their eyes, and pumping white noise into their ears through headphones. Another individual (the "sender”) located in an acoustically shielded room attempts to transmit a specific visual stimulus to the percipient, who then is asked to report all mental imagery that comes to mind. Finally, the percipient is presented with a set of several (typically four) visual stimuli, only one of which is the stimulus viewed by the sender, and asked to rate the extent to which each stimulus matches the mental imagery experienced during the session.

The logic of the Ganzfeld technique relies on the concept of the signal-to-noise ratio. The mental information ostensibly detected by ESP percipients is posited to be an extremely weak signal that is typically obscured by a large number of extraneous stimuli. By placing the percipient in a uniform sensory field, the Ganzfeld technique is hypothesized to decrease the proportion of noise relative to signal and thereby permit investigators to uncover normally weak ESP effects.

With the aid of a statistical technique termed meta-analysis, which permits researchers to quantitatively pool results across a number of studies, Bem and Honorton reported what appeared to be strong, if not convincing, evidence for ESP. The subjects in their meta-analysis obtained overall target “hit” rates of approximately 35 percent, where chance performance would be only 25 percent. Moreover, Bem and Honorton reported several psychologically meaningful predictors of Ganzfeld performance. Subjects who 1) were artistically creative (music, drama, and dance students recruited from the Julliard School), 2) extroverted, 3) had previous ESP-like experiences (but who were “novices,” i.e., had no previous experience as Ganzfeld subjects), 4) had previously studied a mental discipline, such as meditation (but who similarly were novices), and 5) received high scores on self-report indices of emotionality and perceptual orientation to the environment obtained especially high hit rates. In addition, experimental conditions using dynamic visual stimuli yielded higher hit rates than those using static visual stimuli.

Bem and Honorton’s findings, which were widely disseminated in both the popular and academic press, have stirred fresh hopes in the parapsychology community that a truly replicable method of eliciting ESP effects may at last be at hand. Moreover, they have been cited in several popular books, including Dean Radin’s The Conscious Universe, and Courtney Brown’s Cosmic Voyage, as providing very promising, if not conclusive, support for the existence of ESP.

Although some critics, like Ray Hyman, found statistical anomalies in the Bem and Honorton data set suggesting the possible existence of subtle but damaging experimental artifacts (see Hyman, R., Skeptical Inquirer, March/April 1996; and Hyman, R., Psychological Bulletin, 1994), Bem and Honorton’s meta-analysis was regarded by many as offering the most compelling laboratory evidence to date for the existence of ESP.

This is essentially where things stood until a few months ago, when Julie Milton of the University of Edinburgh and Richard Wiseman of the University of Hertfordshire published an updated meta-analysis of thirty recent Ganzfeld studies not reviewed by Bem and Honorton. Milton and Wiseman’s findings, which were published recently ("Does Psi Exist? Lack of Replication of an Anomalous Process at Information Transfer,” Psychological Bulletin 125(4): 387-391), stand in stark contrast to those of Bem and Honorton and raise serious questions concerning the replicability of the Ganzfeld findings. Specifically, Milton and Wiseman reported a mean effect size across all thirty studies of .013, which corresponds to essentially chance performance and can most charitably be described as negligible.

Moreover, Milton and Wiseman failed to replicate Bem and Honorton’s findings that a previous history of ESP-like experiences and the use of dynamic targets predicted enhanced Ganzfeld performance. (Because of insufficient information in the studies, Milton and Wiseman were unable to directly examine Bem and Honorton’s other predictors, such as extroversion.) In contrast, Milton and Wiseman did find that previous participation in a mental discipline among novices predicted enhanced Ganzfeld performance. Ironically, however, a re-examination of Bem and Honorton’s analyses revealed that this predictor was incorrectly identified as statistically significant in their original article, suggesting that the overall findings for the mental discipline variable in fact amount to another replication failure. In the words of baseball hall-of-famer Yogi Berra, Milton and Wiseman’s findings appear to be a case of “déjà vu all over again.” Seemingly replicable parapsychological findings have again proven to be disconcertingly elusive, and the experimental ESP literature has again proven to be consistently inconsistent.

Parapsychologists have already begun to raise questions regarding Milton and Wiseman’s findings and conclusions. For example, some have criticized Milton and Wiseman for including a heterogeneous set of studies in their meta-analysis, and have pointed out that several studies in their database were in fact statistically significant. Nevertheless, Milton and Wiseman reported that a statistical test of homogeneity conducted on the individual effect sizes suggested that the studies in their meta-analysis can be regarded as being drawn from the same overall “population” of studies.

It seems likely that Milton and Wiseman’s meta-analysis will not be the final word on the Ganzfeld technique, and the question of whether this technique will prove to be the replicable paradigm long sought by parapsychologists or merely another tantalizing will-o'-the-wisp is far from conclusively resolved.

It is evident, however, that the ball is now back in the court of parapsychologists, who will need to convince open-minded skeptics that the Ganzfeld technique will not go the way of J. B. Rhine’s classic Zener card studies, Targ and Puthoff’s remote viewing studies, and other superficially promising but ultimately disappointing ESP paradigms. Otherwise, it may soon be back to the drawing board for yet another paradigm.

Scott O. Lilienfeld

Scott O. Lilienfeld is associate professor of psychology at Emory University and Editor of the Scientific Review of Mental Health Practice