Unnatural Acts: Critical Thinking, Skepticism, and Science Exposed!
By Robert Todd Carroll. James Randi Educational Foundation, 2011. Available through Amazon.com and other electronic outlets exclusively in e-book format, $9.99.

Robert Todd Carroll, the author of the invaluable Skeptic’s Dictionary, has written an e-book that makes a perfect complement to his Dictionary. Titled Unnatural Acts: Critical Thinking, Skepticism, and Science Exposed!, it is essentially a primer on how to think.

The “unnatural acts” of the title are the acts of critical thinking, which don’t come naturally to our imperfect human brains. Our brains evolved an instinctive, intuitive, quick-and-dirty way of thinking that served our forebears well in their environment. A slower, more systematic, more critical way of thinking developed later and brought us science. It serves us better in today’s world but is more difficult to achieve. It requires education and concentrated effort to overcome the natural tendencies imposed on us by our evolutionary history.

The brain is an illusionist. It works by taking shortcuts, deceiving us into seeing things that aren’t there and believing things that aren’t true. We see the sun apparently moving across the sky, and it takes sophisticated understanding to overcome our first assumptions. Nature has programmed us to in­crease our chances of survival and reproduction, not to seek the truth. Religious literalists, New Age philosophers, and other true believers prefer magical thinking to science because we are hardwired to think that way. So creationism is often preferred over evolution, and fanciful medical quackeries are often preferred over effective treatments proven by scientific studies.

Critical thinking is hard. We must learn that we can’t trust our perceptions, memories, and intuitions. This requires education and practice. It’s frustrating to accept that our most cherished beliefs might be wrong. And it’s frustrating to know that success is elusive: the truths we learn will always be provisional, and we can never be certain whether we have looked at all the relevant data objectively.

In successive chapters, Carroll ex­plains:

• Critical thinking is unpopular: you will lose friends and alienate your neighbors.
• Trust no one, not even yourself: you, too, are subject to perceptual distortions and cognitive biases.
• Language is often used to manipulate thought and behavior.
• Groupthink—communal reinforcement—seduces groups of people into bad decisions.
• It is becoming increasingly difficult to identify reliable information amid all the hype, propaganda, advertising, and misinformation.
• Anecdotes are compelling: a good story trumps a dozen scientific studies. Scientific studies are more reliable in the search for truth but are also subject to bias.
• Fallacies in reasoning are widespread and natural; man is an irrational animal.
• Natural factors conspire to lead us into error, but there is hope that we can learn to overcome our natural tendencies and become critical thinkers.

In the final chapter, Carroll provides practical advice: fifty-nine ways to develop your unnatural talents in critical thinking, skepticism, and science. This list will be enhanced by the blog Unnatural Acts (www.59ways.blogspot.com), where he will be offering commentary and examples of the fifty-nine ways (plus a few more). Five appendices round out the banquet with detailed discussions of cell phone radiation, interstellar travel, acupuncture, what it really means to think critically, and step-by-step instructions on how to create your own pseudoscience. Refreshingly, he admits to errors of his own in The Skeptic’s Dictionary and corrects the record.

I laughed out loud at Carroll’s account of his first attempt at teaching logic as a newly minted philosophy PhD. “If I remember correctly, about twenty-five students signed up for the class and three finished. One of the three stayed because he liked me. Another stayed because he didn’t know how to drop a class. The remaining student understood the material in the text . . .” Over time, he progressed from teaching traditional logic to offering more useful courses in critical thinking as a way of life, emphasizing an understanding of the psychology of bias and other sources of error and embodying an attitude of intellectual humility, confidence in reason, intellectual curiosity, and intellectual independence.

Since critical thinking is important to every aspect of human life, he illustrates his points with examples drawn from every imaginable field: religions, UFO cults, psychology, alternative medicine, politics, parapsychology, martial arts, criminology, climate change, news media, vaccines, cell phone radiation, cancer clusters, and more.

Some of his examples will be familiar from other skeptical writings, like Clever Hans (the horse that responded to its owner’s unconscious body language) and the basketball/gorilla video illustrating inattentional blindness. Others were new to me, like the example of audio pareidolia where a Bob Dylan song lyric is heard as “throw my chicken out the window.”

There can never be too many books on critical thinking. Carroll’s is a worthy contribution to the skeptical literature: comprehensive, easy to read, and packed with entertaining examples that vividly illustrate the concepts. For those new to skepticism, it can serve as a valuable textbook for learning how to think. It will be useful to even the most jaded skeptics among us who think we already know how to think; we all still make mistakes, we need to be reminded anew of old lessons, and there is always more to learn.

We hear a lot about risks, but the media seldom puts those risks into perspective for us. The swine flu had killed 263 people in the US as of July 10, 2009. Regular flu kills 36,000 each year. Smoking kills 440,000 each year. We should be far more afraid of cigarettes than of swine flu.

We are warned about the obesity epidemic and we are told the ideal body mass index is 19-25. For a BMI of 18 or lower, there are 34,000 excess deaths. For a BMI over 30, there are 112,000 excess deaths. But lo and behold, for BMIs only mildly elevated at 25-29, there are 86,000 FEWER deaths. Makes you wonder what “ideal weight” really means.

We all know how easy it is to lie with statistics. Here are some statistical benchmarks to keep in mind that will help you detect some of the lies. The US population is a little over 300 million, with 4 million births and 2.4 million deaths a year. Half of the deaths are from heart disease and cancer, and you can read the rest. The report that “more than 4 million women are battered to death by their husbands or boyfriends every year” can’t possibly be true because total homicides are only 17,000. 4 million exceeds the total 2.4 million mortality.

Another area of confusion is the difference between prevalence and incidence. Prevalence is how many people in a population currently have the disease; incidence is how many people are newly diagnosed each year. One of my colleagues on the blog was discussing autism rates with an anti-vaccine activist. He cited the incidence figures for Denmark and his opponent tried to compare them to the prevalence figures in the US and ended up with egg on his face.

It’s very important to remember that probabilities are not predictions. Children who consistently spend more than 4 hours a day watching TV are more likely to be overweight. But that doesn’t mean you can make a child fat by making him watch TV. Or that you can make a child thinner by turning the TV off.

When there are a lot of small studies that don’t show statistical significance, sometimes it is possible to combine data to make one big study that does show statistical significance. Usually, the studies are not similar enough to justify doing that. If the small studies are not well-designed, combining them just makes the Garbage In Garbage Out (GIGO) problem worse. Systematic reviews examine all the published data using predetermined criteria and trying to weigh the quality of evidence. Because of the characteristics of a systematic review, if the results are negative they are probably true, but if they are positive they may still have a GIGO problem. Sometimes the reviewers are biased – they may have homeopaths reviewing the literature on homeopathy.

Homeopathy: the perfect placebo, nothing but water. Cartoon: Why homeopathy is such a bargain: just before you run out, use what’s left to mix up another gallon.

A systematic review of homeopathy studies showed that overall it worked better than placebo, but it didn’t work better for any specific condition. Whaaat? That’s like saying broccoli is good for everyone but is not good for men, women or children.

One of my readers educated me about how this can happen, and I thought it was neat so I’ll share it with you. It’s called Simpson’s paradox. Looking across the first row, testing treatments for symptom A, 30 patients were given placebo; 24 improved for an 80% improvement. 50 patients were given a homeopathic remedy and 40 improved, for an 80% improvement. In the rows for symptom B and C the same thing happened: the improvement percentages for placebo and homeopathy were exactly equal. But look what happens in the bottom row. When you add up rows A, B, and C, it looks like you got a 35% improvement with the placebo and a 48% improvement with homeopathy. This is statistical shenanigans, NOT evidence that homeopathy works better than placebo.

Poster of toilet: If water has a memory, then homeopathy is full of shit. Homeopathy: shit and sugar.

If a test is negative, how likely is it that the patient really doesn’t have the disease? If the test is positive, what is the likelihood that he really has the disease?

We can calculate likelihood ratios. Here’s a complex example, where the thickness of the lining of the uterus is measured on ultrasound to predict the likelihood of cancer of the uterus in women who have postmenopausal bleeding. If the thickness is less than 4 mm, the likelihood is 0.2%; if it is 21-25, the probability of cancer is 50%.

Some tests are better at ruling out disease and some are better at ruling it in. There is a blood test called the D-dimer test that is used to help rule out pulmonary embolus. If the doctor thinks there is a 10% probability that the patient has a PE, a positive D-dimer test raises the probability to 17%; a negative D-dimer test lowers the probability to 0.2%.

You have a sore throat. You get a throat culture. If the culture is positive, you have strep throat. If it’s negative, you don’t have strep throat. Right? WRONG! You might be an asymptomatic strep carrier, and your symptoms might be due to a virus. There could be a lab error. False positive and false negative test results are possible.

Rather than depending on a throat culture, we’ve developed some clinical decision rules. We can calculate a strep score based on points for age, exudate on tonsils, swollen lymph nodes, fever, and absence of cough.

If the score is 0-1point, the probability of strep is essentially zero and no culture or treatment is necessary. If the score is 2-3 points, the probability of strep rises to 17-35%, and we get a throat culture and give antibiotics if the culture is positive. If the score is 4-5 points, the probability is over 50% and we can just give antibiotics and not bother doing a culture.

Medicine is not an art or a science, but an applied science. It can be tricky to apply the evidence from published studies to the individual patient in the doctor’s office. Often there are no pertinent studies to go by. We can’t just use blind guesswork or intuition. We have to make the best possible judgment based on the best available evidence.

Medical science is flawed, but it’s better than the alternatives: testimonials, personal experience, belief-based treatments, or hypothetical, untested treatments. Cartoon: “But pastor, I need an antidote, not an anecdote.

If you don’t have a reliable way to evaluate evidence, you can end up being fooled by quackery like the blue dot cure. A quack can do anything useless and ridiculous, like painting a blue dot on the patient’s nose, and can “show” that it works. There are only 3 things that can happen: the patient can get better, worse, or stay the same. If he gets better, the quack claims that the blue dot worked. If he gets worse, the quack laments “If only you’d come to me sooner, the blue dot would have had time to work.” If the patients stays the same, the quack says “The blue dot kept you from getting worse; we need to continue the treatment.”

People are easily fooled into believing that quack treatments work. Cartoon: “Strong and blind belief is a virtue.” “OK then. I will strongly believe that you don’t know much of anything.”

Real science and pseudoscience may be hard for the layman to tell apart, but it’s important to know the difference, because…

Quackery takes your money.

Pseudoscientific explanations may seem to make superficial sense if you don’t know anything about science. Cartoon: Why does the sun set? It’s because hot air rises. The sun’s hot in the middle of the day, so it rises high in the sky.

In the scientific method, the scientist says “Here are the facts. What conclusions can we draw from them?” In the pseudoscientific method (here represented by the creationist method) the pseudoscientist says “Here’s the conclusion. What facts can we find to support it?” The scientist asks IF something works; the pseudoscientist tries to SHOW that it DOES work.

As Bill Nye, the Science Guy, says: Science rules!

Here’s a brief overview of how to read a study. The first thing is the abstract, a summary of what the study showed. It is only as good as the person who wrote it, and it may be inaccurate or it may draw conclusions not warranted by the data. The introduction explains what we already knew about the subject and why they decided to do this particular study to learn more. It usually cites previous studies, and if the authors are biased, it may cherry-pick the literature and mention only studies that support their bias. Then there is a Methods section that should describe in detail exactly what they did, so a reader could replicate their experiment in his own lab. Then they give their Results. They should give us the raw data so we can do our own analysis. This is where they apply statistical tests and tell us whether their results are statistically significant and at what level of significance. The Discussion section tells us what they think their data shows. If they are good scientists, they will point out the limitations of their study and try to anticipate the objections that others might raise. Then with the Conclusion, they will tell us what they think the implications of their study are. A good scientist will usually say something like “If other studies confirm these findings, this treatment may turn out to be clinically useful.” A bad scientist might say something like “We have proved this treatment works and everyone should start using it right away.” The references will be listed, and it’s worth checking them. Sometimes you can tell from the title alone that the reference has little to do with the claim it is meant to support. For instance, the statement “homeopathy cures rheumatoid arthritis” might be referenced by a homeopathy handbook or by a study entitled “The prevalence of a positive RA factor in a population with advanced rheumatoid arthritis.” I have even found references that attested to the exact opposite of the claim made in the article. Then, at the very end, there should be a mention of the source of funding and a disclosure of any conflicts of interest on the part of the authors.

I’m going to go over a couple of examples of how I go about evaluating a study, first from an abstract alone, and then from an entire study. Let’s say you read in the newspaper about a new study that shows that taking a multivitamin will reduce your risk of a heart attack. How can you know if the newspaper report has represented the study fairly? Most studies are listed in PubMed at www.pubmed.gov with their abstracts. You can usually get enough clues from the news report to use the search function and find the abstract. You will see something like this:

Don’t worry about the details. I’m just going to go through and mention some of the kinds of things I look for. The red highlighted phrases caught my attention. This was a study in Sweden. They took 1296 people who had had a heart attack (myocardial infarction or MI) and compared them to 1685 controls picked from the general population and matched to the patients by sex, age, and catchment area. They asked them to self-report their use of diet supplements, and they found that patients who had not had a heart attack were significantly more likely to have used diet supplements. This was a case-control study, which is less reliable than a randomized controlled study. It depended on recall rather than measuring in any way what the patients had actually taken. It did not try to assess whether any of the patients had been vitamin deficient. It only studied patients between the ages of 45 and 70, so any conclusions drawn from it might not apply to people older and younger than that. It was done in Sweden; they themselves point out that the consumption of fruits and vegetables is relatively low in that country, so it might not have implications for other countries with different diets. Of those who said they used supplements, 80% said they used multivitamins. What about the other 20%? Could they have been taking some other supplement that had a larger effect than vitamins? They checked for some possible confounding factors, and they found that “never smoking” outweighed the effect of vitamins in women. The abstract concludes “Findings from this study indicate that use of low dose multivitamin supplements may aid in the primary prevention of MI.” I don’t think the data support that conclusion at all. At any rate, the headline is clearly wrong. This study does NOT show that you should start taking a multivitamin to reduce your risk of a heart attack. More importantly, when you search the medical literature you don’t find any other studies showing that multivitamins prevent heart attacks; in fact, there are several studies showing that supplemental vitamins either have no effect or make things worse.

Now let’s go over an entire study. I picked this one: Dominican children with HIV not Receiving Antiretrovirals: Massage Therapy Influences their Behavior and Development. It was published in an online journal “Evidence-based complementary and Alternative Medicine (eCAM). The very title of the journal suggests bias, since the definition of CAM is that it is not supported by the kind of evidence that would lead to its adoption by mainstream medicine.

In short, they took children who were HIV positive and divided them into two groups, comparing massage therapy to play therapy. Play therapy was the placebo control they chose: you can judge for yourselves whether that is an adequate placebo.

The abstract explains that they studied 48 Dominican children between the ages of 2 and 8. They all had untreated HIV/AIDS. They were randomized to receive either massage or play therapy for 12 weeks. The massage group improved in self-help abilities and communication. Children over the age of 6 showed a decrease in depressive/anxious behaviors and negative thoughts.

The first thing I noticed was a number of careless errors. Caribbean was mis-spelled. Their phrase “for enhancing varying behavioral and developmental domains” was so vague as to be essentially meaningless. They said “A second objective of our work was to determine the absence of antiretroviral treatment on the impact of HIV infected Dominican children’s mood and behavior.” The abstract said the sessions lasted 20 minutes, while the text said 30 minutes. By themselves, errors like these may not mean much, but it makes me wonder if the researchers were as careless about their experimental methods as they were about writing up the results. It also makes me wonder what the peer reviewers and editors were doing – they certainly weren’t doing their job, or they would have caught and corrected errors like these.

In the Introduction section they offered a brief review of the literature. They summarized a few cherry-picked studies that didn’t really support their claims, they omitted other studies that contradicted their claims, and they didn’t address plausibility. The rationale for doing this study was unclear. They talked about massage enhancing immune function, but this study did not even try to measure immune function. It only looked for psychological and developmental effects.

In the Methods section, they described 30 minute sessions with either a standardized massage protocol or a play therapy protocol. The play therapy consisted of giving the child a choice of coloring/drawing, playing with block, playing cards, or reading children’s books. The parents were present throughout, and the parents’ reports were used to judge improvement.

They used a couple of scales to measure improvement. This one, the CBCL, measured these 8 items, and another scale measured several others. So we have a multiple end point problem and there is no evidence that they tried to correct for this.

The Results section reported that for children under 5 there were no significant differences between the massage and play groups. For children over 6, they reported “significant” improvement in the massage group for anxious/depressed behavior, negative thoughts, and overall internalizing scores. Look at the p values. The value for negative thoughts is p=0.059, higher than the usual cut-off of 0.05. Most scientists would not report this as “significant.” And note that among all the many things they measured, only two showed true statistical significance.

They seemed to be confused about the meaning of significance. They reported that 100% of the children in the play group showed an increase in their score on rule breaking behaviors, significant at a p=<0.05 level. They commented that “this significant change was not clinically meaningful.” How could they determine that? And if that wasn’t clinically meaningful, how did they determine that their other findings WERE clinically meaningful? They characterized a change in IQ data as “marginally significant for the massage group at p=0.07. That’s like being a “little bit pregnant.” The cut-off for significance is p=0.05, and most researchers would simply call anything over the cut-off not significant.

On the Developmental Profile scales, they found that the massage group improved in self-help and communication. The play group improved in social development, while the massage group showed decreased social development. They found no significant differences in the physical or academic scores. There is a bar graph printed in the article that appears to show that the play group did better with a gain of 9 points compared to 8 for the massage group.

In the Discussion section they said that “massage therapy was effective in reducing maladaptive internalizing behaviors in children aged 6 and over” and “children 2-8 years of age who received massage demonstrated enhanced self-help and communication skills.” They found it “interesting” that children in the massage group remained at the same social developmental level, suggesting that it was because those children had little or no play activity at home. (Did they? We don’t know.)

They were “puzzled” by their failure to find any effect on behaviors in the under-5 age group because they were so sure massage therapy improves children’s moods and anxiety levels. They tried to rationalize what might have gone wrong. They commented that “anecdotally, the nurses who conducted the massages reported changes in the children over time, including better mood.” This kind of anecdotal report is meaningless and has no place in an objective scientific study. It is a blatant attempt to make massage look better than what the data showed.

They recommended massage therapy as a cost-effective option to improve symptoms and functioning in children with untreated HIV. Sorry, but the study doesn’t even begin to support such a recommendation.

Why was this study really done? We already know that children need human interaction, play, touch, and TLC. Why massage? Does this study justify using massage as a “band-aid” on children who are denied life-saving anti-AIDS drugs?

I could have taken the data they collected and used it to support a very different conclusion: We studied a bunch of outcomes and found that massage is ineffective for all but a couple of them (and those are probably not clinically meaningful). One outcome was worse with massage (decreased social development).We did not show that massage is any better than TLC. Money would be better spent saving lives with effective drugs.

Cartoon: clown says “It may surprise you to hear that, actually, morphine is the best medicine.”

In the syllabus we have reprinted one top-of-the-line study and one worthless study. In your group discussion sessions, I’d like you to compare the two. Some of it is technical: don’t worry about the parts you don’t understand. Look for reasons one is better than the other.

The end.

Title slide

Many of you know I wrote this book, entitled “Women Aren’t Supposed to Fly: the Memoirs of a Female Flight Surgeon. I thought I would share with you the REAL reason women aren’t supposed to fly.

That’s her purse hanging from the instrument panel.

This workshop is called the Skeptic’s Toolbox, and every year we talk about what makes some people skeptics and others not. You can take two people who believe in dowsing and explain the ideomotor effect and show how dowsing has consistently failed every properly controlled test. One person will accept the evidence and stop believing in dowsing. The other will ignore the evidence and continue to believe. Why is that? Bob Carroll, author of The Skeptic’s Dictionary has said that critical thinking is an unnatural act. Science doesn’t come naturally.

Jerry Andrus used to come to the Toolbox and show us his close-up magic tricks and optical illusions. He would tell us “The reason I can fool you is that you have a wonderful brain.” Our brain takes the odd contraption on the right, and from another point of view on the left it assembles it into a nonexistent, impossible box.

Psychiatrist Morgan Levy said, “Thinking like a human” is not a logical way to think but it is not a stupid way to think either. You could say that our thinking is intelligently illogical. Millions of years of evolution did not result in humans that think like a computer. It is precisely because we think in an intelligently illogical way that our predecessors were able to survive.

He also said “Scientists expend an enormous amount of time and energy going to school in order to learn how to undo the effects of evolution so that they can investigate natural phenomena in a logical way.” Education helps, but it isn’t enough. We all know highly educated people who are not skeptics.

Ray Hyman has suggested that skeptics are mutants. Has something new evolved in our brains to help us overcome our intelligently illogical thinking processes? Well, Ray, I’m here to tell you you are right. Skeptics ARE mutants, and I have proof.

This is a picture of my daughter Kimberly.

(Title Slide again). But to get back to the subject of my talk… I’m going to explain some of the pitfalls we encounter when we apply the scientific method to clinical medicine. I’m going to try to give you a feel for how I evaluate a study to see if it is credible. I’m going to suggest some questions you may want to ask the next time you visit your doctor. This is the Skeptic’s Toolbox, and I’m hoping to offer you some tools so that the next time you see a report in the media claiming that broccoli causes cancer you will have a better handle on how to evaluate the report, what questions to ask, and how to decide whether you should immediately stop eating broccoli.

Before we had science we had to rely on these things. Folk remedies, old wives’ tales, herb women, witch doctors, superstitions. The plague doctor in the upper left picture wore a beak-like mask stuffed with herbs in the belief that it would keep him from catching the disease. Instead of scientific trials we had only case reports and testimonials – for centuries we used fleams like the one in the upper right picture for bloodletting and we kept doing it because both patients and doctors told us it worked. The proclamations of authorities like Hippocrates and Galen were never tested or questioned, and their errors were passed down for centuries.

Eventually, we crossed the line from proto-science to modern science, for instance from alchemy to chemistry: Cartoon: “You’ve turned lead into gold? Good. Do it again, write a detailed description of how you did it, and submit it to peer review.”

Here’s a short history of medicine: 2000 BC: eat this root. 1000 AD: that root is heathen, say this prayer. 1850 AD: that prayer is superstition, here, drink this potion. 1920 AD: that potion is snake oil, here, take this pill. 1965 AD: that pill is ineffective, here, take this antibiotic – and then…

Back to square one: 2000 AD: that antibiotic is artificial, here, eat this root. [Cartoon: Natural remedies, snake oil.]But that’s only in some circles. Mostly we stick to science-based medicine today.

Scientific method is nothing special; it’s really just a way of thinking about a problem, forming a hypothesis, selecting one variable at a time, and testing it. It is the only discipline that allows us to reliably distinguish myth from fact. It can be as simple as trouble-shooting to find out why the radio isn’t working. Hypothesis: it’s not plugged in. Test: check the plug. And so on.

We have gradually learned better ways to test. The picture on the left is of Don Quixote. In the novel, he is assembling some old armor to wear on his quest and discovers that the visor is no longer attached to the helmet. He attaches it with ribbons, and when he tests it by striking it with his sword, the visor falls off. He re-attaches it more carefully and the second time he decides he doesn’t really need to test it again. Modern science has developed more rigorous ways to do tests.

Hard science is easy, soft science is hard. When you mix chemicals A and B you always get chemical C, and you can calculate exactly how much will be produced. But medical studies are more problematic. Every molecule of a chemical is the same, but humans are not all the same. We have genetic differences, and there are confounders like age, diet, alcohol, and concurrent diseases that may all influence the response to a treatment. Subjects in a study may not take all their assigned pills. In medical studies, A plus B may appear to equal C or D or E.

The first modern clinical trial was done by James Lind of the Royal Navy in 1747. Back then, the sailing ships went out for years at a time and sailors had no access to fresh foods. Many of them developed scurvy, where they became weak, unable to work, and had internal bleeding, bleeding gums, and other symptoms. Today we know this was due to a deficiency of vitamin C. But vitamins weren’t discovered for another 2 centuries. Lind had heard reports of successful treatment, and he had developed the hypothesis that scurvy was due to putrefaction and could be prevented by acids. He divided 12 sick sailors into 6 groups, kept them all on the same diet, and gave each group a different test remedy: a quart of cider, 25 drops of elixir of vitriol (sulfuric acid – I hope he diluted it in water before he gave it to them!), 6 spoons of vinegar, half a pint of seawater, two oranges and a lemon, or a spicy past plus a drink of barley. The winning combination was two oranges and a lemon. It worked, but he still didn’t understand HOW it worked. He tried sending ships out with bottled juice to save storage space, and that didn’t work – the bottling process heated the juice and destroyed the vitamin C.

Applying scientific method to medicine led to lots of great discoveries. Vaccines are probably responsible for saving more lives than anything else. Reliable birth control allowed women to take control of their lives and contribute to society in all kinds of occupations. Antibiotics reduced the toll of infections. We developed fantastic imaging methods (x-rays, ultrasound, CT, MRI, PET scans) (that’s Homer Simpson’s skull x-ray) that enabled us to see inside the living body and make diagnoses without waiting for the autopsy. Diabetes used to kill all its victims: insulin keeps them alive today. We can even transplant organs. And, since I mentioned birth control pills for the women, I’ll mention Viagra for the men. Some men probably think that’s one of medical science’s greatest inventions.

Modern medicine has accomplished a lot. Cartoon: “These machines sure are live-savers, doc. The noise annoyed me right out of my coma.”

But the greatest discovery of all was probably the randomized controlled trial (RCT). It was the method that enabled us to make most of those great discoveries.

The R in RCT is randomization. When comparing the responses of two groups, you want to make sure the groups are comparable. If you steered all the sick, old people into the treatment group and all the young, healthy ones into the placebo group, an effective treatment might appear to be a dud. To make sure there is no bias in group assignment, it’s best to use concealed allocation, where the researcher that assigns patients to groups 1 and 2 doesn’t know which is the placebo group; only another researcher knows that. Once you have assigned patients randomly to the groups, you still need to go back and check that the two groups really are similar.

In reports of well-designed studies, you will usually see a table like this.

Here’s a close-up. The details aren’t important. The point is that they looked at all kinds of parameters like age, sex, ethnic group, weight, height, blood pressure, heart rate, etc. and calculated statistical measures of how similar the groups were.

The second letter of RCT stands for Controlled. Treatments almost always “work” – even quack treatments often seem to work due to the placebo effect and due to the natural course of disease where some people improve without treatment. There is a phenomenon called the Hawthorne effect, where just being enrolled in a study leads to improvement. So instead of just treating subjects and showing that they improve, you need to compare them to a control group of patients who get a placebo, or a known effective treatment, or no treatment at all.

Here’s why that’s important. Almost any disease has a fluctuating course. Symptoms wax and wane unpredictably. Here’s an example of a woman with osteoarthritis pain in her knees. At the beginning of this particular month she hardly had any pain, then it got worse for a while, when it subsided again. This is what it does with no treatment at all. Let’s say she decides to take a pill for the pain. She’s not likely to try it during the first few days when the pain is hardly noticeable. She’s more likely to try it when the pain peaks. And look what happens.

Her pain goes away! Of course this is exactly the same as the previous graph with the left half cut off. But it sure is convincing. It really looks like whatever she did worked.

Now let’s say she gets a placebo effect from the treatment. The pain was going away anyway, but now it goes away really fast. Of course she’s going to conclude that the treatment worked wonders. So if you want to prove that a new treatment works, you’re going to have to show that it produces more improvement than the natural course of disease and the placebo response.

The best RCTs are double blind. The subject doesn’t know whether he’s getting the treatment or the placebo, and the researcher doesn’t know which he’s giving the patient. This minimizes any unconscious effects of bias. But even double blinding isn’t foolproof. Sometimes patients can guess which group they were in due to side effects. Sometimes they even take the capsules apart and check whether the contents taste like sugar. So you really need to do an exit poll, asking patients which group they thought they were in. If they can guess better than chance, you didn’t have an adequate placebo and your results are tainted.

Now let’s look at how these RCTs are used in practice to develop new drugs. Cartoon: “Just for kicks, let’s come up with something that has a good side effect.”

The first step is to decide what’s worth testing. If an Amazon explorer reports that a tribe chews a certain leaf to treat infections, you don’t just bring those leaves home and give them to people. First you might test it in a lab to see what the components were and see whether they suppressed growth in a bacterial culture. You might try it on animals with infections. In vitro is Latin for “in glass” and refers to lab testing with test tubes and Petri dishes. In vivo means in a living body.

Animal studies may not be valid. Cartoon: “I believe I have a new approach to psychotherapy, but, like everything else, the FDA tells me it first has to be tested on mice.” Animals are not always equivalent to humans. Aspirin would have been rejected on the basis of animal studies. It causes congenital defects in mice, but not in humans.

Once you’ve decided that a drug is worth testing, the next step is what we call Phase I trials. It looks for toxicity to determine if the drug is safe. You give a single small dose of the drug to healthy volunteers. If they have no adverse effects, you test larger doses over longer periods. Sometimes there are unpleasant surprises. A drug called TGN1412 monoclonal antibody was tested in animals and appeared to be safe. They took a dose 500 times lower than was found safe in animals and gave it to 4 healthy volunteers. They were all hospitalized with multiple organ failures and some required organ transplants to save their lives.

If the drug passes the Phase I safety trials, the next step is a Phase II trial to see if the drug is effective. Phase I is the first trial in humans; Phase II is the first trial in patients. It tests large numbers of patients with the target disease and compares different doses. Ideally, several trials are done with different patient groups.

If the Phase II trials show the drug works, the next step is a Phase III trial to see if it works better than other treatments. You compare the new drug to an older drug or to a placebo.

Placebo trials may not be ethical. Cartoon: “Half the diabetics were given the new drug and responded well. The other half got a placebo and went into shock.” If you have an effective treatment for a disease, you can’t risk patient’s lives by denying them that treatment and assigning them to a placebo group. We couldn’t do a trial today comparing appendectomy to a placebo for acute appendicitis.

The trials don’t end with approval and marketing. They continue with post-marketing Phase IV trials to see what more we can learn. The company may want to demonstrate that the drug works for other illnesses, or show that it works better than a competitor’s product.

And then there’s post-marketing surveillance. The number of subjects in a research trial is small compared to the number of patients who will be taking a drug after it is marketed. Some of them will be different from the people in the trials in various ways, for instance they may have concomitant diseases. The first rotavirus vaccine was taken off the market when they discovered that 1 in 10,000 children developed intussusception, a telescoping of the bowel that is life-threatening. 1 in 100,000 people who got the 1976 swine flu vaccine developed a paralysis called Guillain-Barré syndrome. How many people would you have to enroll in a premarketing study to detect a one-in-100,000 complication? One of the weirdest drug effects I came across was that in men taking a drug called Flomax for prostate symptoms, if they have cataract surgery they can develop a complication called “floppy iris syndrome.” No one could have predicted that!

A researcher named Ioannidis recently wrote a seminal paper showing that most published research findings are wrong. And he explained why.

There are any number of things that can go wrong in research. Here are some of them. If a drug company funds the research, it’s more likely to support their drug than if an independent lab does the study. If the researchers are true believers, all kinds of psychological factors come into play and even if they do their best to be objective, they are at risk of fooling themselves. People who volunteer for a study of acupuncture are likely to believe it might work; people who think acupuncture is nonsense probably won’t sign up. Maybe 3 studies were done and only one showed positive results and that’s the one they submitted for publication (the file drawer effect). Most researchers delegate the day-to-day details of research to subordinates. Sometimes the peons in the trenches are just doing a job and trying to please their boss. They may feed false data to the author or suppress information they know he doesn’t want to hear. Sometimes when you read the conclusion of a study and go back and look at the actual data, the data don’t justify the conclusion. The report can’t possibly contain every detail of the research – what are they not telling us? Were there a lot of dropouts? Maybe when it didn’t work, they quit, and only the ones who got results were left to be counted.

Here are just a few of the other things that can go wrong. Some countries only publish studies with positive results. In China, if you published a study showing something didn’t work you would lose face and lose your job. So I won’t trust any results out of China until they are confirmed in other countries. If there are only a few subjects, errors are more likely. If you study the net worth of 5 people and Bill Gates is one of the 5, you get skewed results. In general, the more subjects, the more you can trust the results. When they calculate the statistics, they can use the wrong method or make mistakes. They can misinterpret the findings. The file drawer effect is when negative studies are not submitted for publication; publication bias is when the journals are less likely to publish negative studies. Inappropriate data mining is when the study doesn’t show what they wanted, and they look at subgroups and tweak the data every which way until they get something that looks positive. Sometimes researchers outright lie and commit fraud to further their careers. Sometimes they get caught, sometimes they don’t.

In thinking about a study, there are several things to consider. Was the endpoint a lab value or a clinical benefit? The diabetes drug Avandia decreased the levels of Hemoglobin A1C, a blood test indicating that the disease is under control. Unfortunately, it increased the mortality rate. Good blood tests aren’t very important if you’re dead. We try to look for POEMS – Patient Oriented Evidence that Matters. Instead of looking at cholesterol levels, we look at numbers of heart attacks. What kind of study was done? RCT, cohort, case-control, epidemiologic? Was it blinded? Was the study well-designed? Did they use an intention to treat analysis to correct for dropouts?

How much background noise was there? If you look at enough endpoints you’re almost certain to find some correlation just by chance. In a study of Gulf War Syndrome they looked at veteran’s wives to see if the husbands had brought anything home to harm them. They found that wives were less likely to have moles and benign skin lesions. Obviously, that was just noise, not a sign that Gulf War Syndrome improves the skin of spouses. Small effects (a 5% improvement) are not as trustworthy as large effects (60% improvement). It’s always better if different types of evidence from different sources arrive at the same conclusion. You should never trust a single paper, but should look at the entire body of published evidence. And you should trust empirical papers which test other people’s theories more than empirical papers which test the author’s theory.

I want to mention the null hypothesis because it can be hard to understand – it has always bothered me because it seems sort of like a double negative. Instead of testing the claim itself – that X cures Y – you test the null hypothesis – that X doesn’t cure Y. There are only 2 options: you can reject the null hypothesis (more people are cured with X than with placebo) or you can not reject it (the same number of people are cured with X as with placebo). You can’t ACCEPT the null hypothesis, because it’s hard to prove a negative. If the null hypothesis is that there are no black swans, all it takes is one black swan to disprove it. No matter how many times you try to fly, you can never prove that you can’t fly. The more times you try it, the less likely, but science can never say absolutely never. It can only say that based on current evidence, the likelihood of a human being able to fly is so vanishingly small that no one in his right mind would jump off a cliff to try it. Of course, it remains open to new evidence and would be willing to reconsider if people could actually show that they could fly.

All this is very discouraging. If most published research is wrong, when CAN you believe the results? You can look for good quality studies published in good journals, studies that have been confirmed by other studies, studies that are consistent with other knowledge, and studies that have a reasonably high prior probability. As Carl Sagan said, extraordinary claims require extraordinary proof.

Look at this structure. I think you can see that the prior probability of your being able to take 12 dice and construct this is so low that you would not waste your time trying.

Clinical research usually uses the arbitrary level of p=0.05 as the cut-off for statistical significance. Research in physics demands much more. The p=0.05 level essentially means that if you repeat the trial 20 times, 19 are likely to show the same result. Or if something doesn’t work, it might appear to work in 1 out of 20 studies. Does this mean that a study that is statistically significant at p=0.05 is 95% likely to be correct? NO! It may be much less likely to be correct if there is low prior probability. To understand why, let’s look at Bayes’ Theorem.

Bayes’ theorem allows us to use prior probability to calculate posterior probability. Suppose there is a co-ed school having 60% boys and 40% girls as students. The female students wear trousers or skirts in equal numbers; the boys all wear trousers. An observer sees a (random) student from a distance; all the observer can see is that this student is wearing trousers. What is the probability this student is a girl? The correct answer can be computed using Bayes' theorem. (1 in 4)

Here’s a table showing what this means for clinical research. In the top row, if the p-value is 0.05 and the prior probability is 1%, the probability that the results are correct is only 3%. If the prior probability is 50%, the posterior probability is still only 73%. In the bottom row, if the p-value shows a phenomenally high significance level of 0.001, and the prior probability is 1%, the posterior probability is still only 50%. Most of us think the prior probability of homeopathy studies is well under 1%. This shows why the homeopathy studies that claim statistical significance can’t be trusted.

If you don’t consider prior probability, you can end up doing what I call Tooth Fairy Science. You can study whether leaving the tooth in a baggie generates more Tooth Fairy money than leaving it wrapped in Kleenex. You can study the average money left for the first tooth versus the last tooth. You can correlate Tooth Fairy proceeds with parental income. You can get reliable data that are reproducible, consistent, and statistically significant. You think you have learned something about the Tooth Fairy. But you haven’t. Your data has another explanation, parental behavior, that you haven’t even considered. You have deceived yourself by trying to do research on something that doesn’t exist.

Ray Hyman’s categorical directive: “Before we try to explain something, we should be sure it actually happened.” Hall’s corollary is “Before we do research on something, we should make sure it exists.”

There’s been a lot of research on the meridians and qi of acupuncture. Here’s an early acupuncture patient. Cartoon: mammoth is getting spears in the butt and thinks his neck pain suddenly feels better.

In acupuncture fairy science, the hypothesis is that sticking needles in specific acupuncture points along acupuncture meridians affects the flow of qi, which improves health. There is no evidence that specific acupuncture points or meridians exist, and no evidence that qi exists. If it did exist, there’s no reason to think it could flow, or that sticking needles in people could affect that flow, or that the flow could improve health. With acupuncture, what could you use as a placebo control? People generally notice when you stick a needle in them, so blinding is very difficult. They have come up with ingenious placebos – comparing random points on the skin to traditional acupuncture points or using sham needles that work like those stage daggers, where they appear to be penetrating the skin but actually just retract into the handle. One recent study used toothpicks that pricked but did not penetrate the skin. No matter what control you pick, a double blind study is impossible, because the acupuncturist has to know what he’s doing. The best studies using sham acupuncture controls have consistently shown that acupuncture works better than no treatment, and that sham acupuncture works just as well as real acupuncture. It doesn’t matter where you stick the needle or whether you use a needle at all. The only thing that seems to matter is whether the patient believes he got acupuncture. The acupuncture fairy believer’s conclusion is that we know acupuncture is effective so sham acupuncture must be effective too. The rational conclusion is that acupuncture works no better than placebo.

I write for the Science-Based Medicine blog, where we make a distinction between evidence-based medicine and science-based medicine. Evidence based medicine simply looks at the published clinical research and accepts the findings. Science based medicine considers preclinical research, prior probability, consistency with the rest of the body of scientific knowledge, and the fallibility of most research. I like to think of it as this formula: SBM = EBM + CT (critical thinking)

One of the pitfalls in evaluating published studies is disease clusters. If the mean incidence of cancer is X, that means some communities will have fewer than X cases and some will have more than X. If you spill rice on a grid, there will be an average number of grains per square, but some squares will have 0 grains and some will have lots of grains. It can be very tricky to determine whether a cluster is due to chance or whether it represents an increased risk in that particular area. In the Love Canal incident, a panel of distinguished doctors recently reviewed the scientific findings to date. They issued a surprising verdict. In their view, no scientific evidence has been offered that the people of Love Canal have suffered "acute health effects" from exposure to the hazardous wastes, nor has the threat of long-term damage been conclusively demonstrated.

Another tricky thing is confidence intervals. The top of the columns shows the actual measurements found, and there is a 95% confidence that the true measurement falls somewhere on the red lines. In the example on the left, the lowest end of the red line for the brown column is still higher than the highest end of the red line for the white column, so we can be confident brown is really greater than white. In the example on the right, a level within the red line for the brown column is taller than a level within the red line for the white column, so it’s possible that brown might actually be greater than white.

One of the most common human errors is forgetting that correlation doesn’t mean causation. The rise in autism was correlated with a rise in the number of pirates, but I doubt if anyone thinks pirates cause autism or autism causes pirates.

The logical fallacy here is post hoc, ergo propter hoc: assuming that because B follows A, it was caused by A. It’s easy to see why this is a fallacy when you show that the rooster crows every morning, followed by the sun rising. There is a consistent correlation, but we all know it was not the rooster’s crowing that made the sun come up. But think about this: I took a pill, and I got better; therefore the pill made me better. Suddenly it seems perfectly reasonable – but we have to remember that it might be just another rooster.

Does this make sense? It’s from the Scientific American in 1858. A doctor studied the curative effects of light in hospitals, and found that 4 times as many patients were cured in properly lighted rooms than in darkness. He said this was “Due to the agency of light, without a full supply of which plants and animals maintain but a sickly and feeble existence.” The editors commented that the health statistics of all civilized countries had improved in the preceding century – may be because houses are better built to admit more light. I think most of us can see that this correlation did not prove causation – we can come up with other explanations.

A scientist named Hill came up with a list of criteria to determine whether a correlation from epidemiologic studies showed causation. I’ll illustrate with the example of smoking and lung cancer. There is no way we could do a randomized controlled study of smoking – you can’t divide children in two groups and make one group smoke for decades and the other not, and you certainly couldn’t have a blinded study. So we had to approach the question by other routes. 1. There was a temporal relationship: people smoked first and got lung cancer later. 2. There was a strong relationship – lots more smokers than nonsmokers got lung cancer. 3. There was a dose-response relationship – the more cigarettes smoked, the higher the rate of lung cancer. 4. The results of various kinds of studies were all consistent. 5. The mechanism was plausible: we know there are cancer-causing compounds in cigarette smoke. 6. Alternate explanations were considered and ruled out. 7. They did experiments where they exposed lab animals to cigarette smoke and the animals developed cancer. 8. Specificity: cigarettes produced specific types of lung cancers, not a mixture of various unrelated symptoms. 9. Coherence. The data from different kinds of epidemiologic and lab studies and from all sources of information held together in a coherent body of evidence.

Cartoon: Tobacco Industry Research Centre: “Excellent health statistics – smokers are less likely to die of age related illnesses.”

Media reports of medical studies usually make findings sound bad by citing relative risk reduction rather than absolute risk reduction. Rather than knowing percentages, we want to know how many people are harmed or helped.

Here’s an example. A study showed that the risk of breast cancer increases by 6% for every daily drink of alcohol. The relative risk is 6%. So would 2 drinks a day give 12% of women breast cancer? NO. 9% of women get breast cancer overall. In every 100 women, 9 will get breast cancer. If every woman had 2 extra drinks a day, 10 of them would get breast cancer. Only 1 woman in 100 would be harmed; the other 99 would not be harmed. The absolute risk is 1 in 100, not 12.

Another example. Eating bacon increases the risk of colorectal cancer by 21% (relative risk). 5 men in 100 get cancer in their lifetime. If each ate a couple of slices of bacon every day, 6 would get cancer – only one more man, for an absolute risk of 1 in 100.

A recent study showed that using a cell phone doubled the risk of acoustic neuroma (a tumor in the ear). The relative risk was reported as 200% and alarmed parents took their children’s phones away. But the baseline risk of acoustic neuroma is 1:100,000. 200% of 1 is 2. the absolute risk was 1 more tumor per 100,000 people. Acoustic neuroma is a treatable, non-malignant tumor. The lead researcher said she would rather accept the risk and know where her kids were. She let them keep their cell phones. She warned that the results were provisional, the study small, and that different results might be found with a larger study. She was vindicated when a later, larger study found no increased risk.

What’s wrong with this? A British Heart Foundation press release said “We know that regular exposure to second-hand smoke increases the chances of developing heart disease by around 25%. This means that for every four non-smokers who work in a smoky environment like a pub, one of them will suffer disability and premature death from a heart condition because of second-hand smoke.”

If 4 in 100 nonsmokers have heart disease, a 25% increase means 5 will have it. The risk is not 1 in 4, but 1 in 100.

As well as asking for absolute risk rather than relative risk, we can ask for NNT and NNH – the number needed to treat and the number needed to harm. When you use Tylenol for post-op pain, you have to give it to 3.6 patients for one to benefit. You have to treat 16 dog-bite patients with antibiotics for one to benefit: 15 out of 16 will take the antibiotics needlessly. The clot-buster drug tPA has to be given to 3.1 patients for one to benefit, and for every 30.1 patients, one will be harmed.

Lipitor is one of the statin drugs used to lower cholesterol and prevent heart attacks and strokes. When it is used for secondary prevention (in patients who already have heart disease) somewhere between 16 and 23 patients must be treated for one to benefit. When it is used for primary prevention (in patients who are at risk but don’t yet have heart disease) the NNT rises to somewhere between 70 and 250, depending on age, other risk factors, etc. Of every 200 patients taking the drug, one is harmed.

One cynic put it this way "What if you put 250 people in a room and told them they would each pay $1,000 a year for a drug they would have to take every day, that many would get diarrhea and muscle pain, and that 249 would have no benefit? And that they could do just as well by exercising? How many would take that?" This is an exaggeration, but it illustrates that these drugs should be used selectively, based on individual factors like a total risk assessment and the patient’s personal preferences to take his chances vs taking a drug for insurance.

We all tend to assume that a positive test means someone has a disease and a negative test means he doesn’t. But it’s not that simple. There’s no such thing as a definitive test. There are false positives, false negatives, and lab errors. The diagnostic value of a test depends on the pre-test probability that the patient has the disease. One lesson I learned over and over in my years of practice was never to believe one lab test. My mother was a case in point. On the basis of one sky-high blood glucose test, she was told she had diabetes. They wanted to start her on treatment, but I persuaded them to wait. We bought a home monitor and checked her regularly and never ever got a single abnormal reading. When the lab checked her again, she was normal. We still don’t know what happened – maybe her blood sample got switched with someone else’s.

If your mammogram is positive, how likely is it that you actually have breast cancer? They’ve done surveys asking this question, and most laypeople and even many doctors guess 90%. Actually it’s only 10%.

Mammograms are 90% accurate in spotting those who have cancer (this is called the sensitivity of the test). They are 93% accurate in spotting those who don’t have cancer (this is called the specificity of the test). 0.8% of women getting routine mammograms have cancer (this is the prevalence of the disease).

this means that of every 1000 women getting mammograms, 8 of them have cancer. Of those 8 women with cancer, 7 of them will have true positive results, and one will have a false negative result and be falsely reassured that she does not have cancer. 992 of the 1000 women do not have cancer. Of those, 70 will have false positive results and 922 will have true negative results. So in all, there will be 77 positive test results, and only 7 of those will actually have cancer – roughly 10%.

It gets worse. How many lives are saved by mammography? If 1000 women are screened for 10 years starting at age 50, one life will be saved. 2-10 women will be overdiagnosed and treated needlessly. 10-15 women will be told that they have breast cancer earlier than they would otherwise have been told, but this will not affect their prognosis. 100-500 women will have at least one false alarm, and about half of them will undergo a biopsy they didn’t really need.

It gets worse when you screen for multiple diseases. The PCLO Trial tested for cancers of the prostate, colon, lung, and ovary. After only 4 tests, 37% of men and 26% of women had false positives. After 14 tests, it went up to 60% and 49%. After 14 tests, 28% of men and 22% of women had undergone invasive tests to determine whether they really had cancer – this included biopsies, exploratory surgery and even hysterectomy.

Continue to Part 2 »

The Internet is a wonderful medium for communicating ideas and information in a rapid, interactive way. Many online articles are followed by a section for comments. Like so many things in this imperfect world, comments are a mixed blessing. They can enhance the article by correcting errors, adding further information, or contributing useful thoughts to a productive discussion. But all too often the comments section consists of emotional outbursts, unwarranted personal attacks on the author, logical fallacies, and misinformation. It provides irrational and ignorant people with a soapbox from which to promote prejudices and false information.

To illustrate, let's look at some responses to a piece I wrote about a weight-loss product called Isagenix, which is sold through a multilevel marketing (MLM) scheme. To quote its website verbatim, “The Isagenix cleanse is unique because it not only removes impurities at the cellular level, it builds the body up with incredible nutrition. Besides detoxing the body, Isagenix teaches people a wonderful lesson that they don't need to eat as much as they are accustom [sic] to and eating healthy choices are really important and also a lot of the food we are eating is nutritionally bankrupt.”

I didn't set out to write this article. It started when I received an e-mail inquiry about Isagenix. I posted my answer on a discussion list, and it was picked up and published at http://healthfraudoz.blogspot.com/2006/11/critique-of-isagenix.html. Sandy Szwarc, author of a blog titled Junkfood Science, approved of it and kindly reposted it (see http://junkfoodscience.blogspot.com/2006/11/can-you-really-cleanse-your-way-to.html.)

As I write, the comments on the healthfraudoz website have reached a total of 176. A few commenters approved of what I wrote, but the majority tried to defend Isagenix. Their defense was irrational, incompetent, and sometimes amusing.

It was as if no one had actually read what I wrote. No one bothered to address any of my specific criticisms. No one even tried to defend Isagenix's false claims that toxicity accounts for most disease, that the body protects itself from toxins by coating them with fat, and that internal organs become clogged and deteriorate if you don't cleanse. No one offered any evidence that “detoxification” improves human health. No one tried to identify any of the alleged toxins or show that they are actually removed. No one tried to provide any rationale for the particular combination of ingredients (all 242 of them!) in Isagenix products. No one questioned my assertion that “no caffeine added” is inaccurate labeling because green tea, which is added, contains caffeine. No one commented when I observed that the amount of vitamin A in these products is dangerous and goes against the recommendations of the Medical Letter. No one offered any evidence that more weight is lost by adding Isagenix to a low-calorie diet and exercise. I offered some alternative explanations that might account for people believing that Isagenix is effective when it isn't; no one commented on that.

The medical advisor on the Isagenix website argued that at five dollars per day, Isagenix is less expensive than open-heart surgery. I pointed out that this is a laughable false dichotomy: good health is not a matter of choosing between open-heart surgery and diet supplements. No one commented on that. Instead of rational responses, we got:

Testimonials

The greatest number of comments were testimonials: “I took it and I lost weight.” People claimed not just weight loss but a variety of improvements. Isagenix allegedly cured fibromyalgia, osteoarthritis, and hemorrhoids. It facilitated getting off sleeping pills and caffeine, balanced brain chemistry (what does that mean?), improved focus and mental clarity, allowed running longer marathons with less fatigue, saved a failing marriage, stopped irritability and crankiness, and kept an arm from getting sore after pitching.

“Made my son interact appropriately with peers, take care of himself, and want to be hugged and kissed,” claimed one.

“I made money selling it,” said another.

One person wrote, “My out-of-control Irritable Bowel Syndrome disappeared and I had the healthiest BM in about 6 years!... You can't brainwash POO!!”

Two people commented that the Isagenix program provides motivation; one said he needs “structer” (structure?) to stay on a diet.

The plural of anecdote is not data. Two commenters appropriately objected to all this testimonial evidence. They pointed out that testimonials are unreliable and subject to post hoc ergo propter hoc error, that all the “it works for me” comments can be attributed to a low-calorie diet and exercise, and that the testimonials are almost exclusively from people who are selling the product.

Anti-Testimonials

Quite a few commenters reported that they had tried Isagenix and it either didn't work or caused side effects, such as five days of violent diarrhea. One reported gaining a lot of weight while taking it; many reported losing weight just as well without it. Several reported credit-card disputes with the company and failure to get their money refunded. One reported that his parents are using Isagenix and it seems to be slowly killing them: they have decreased energy, declining health, mood swings, and poorer control of diabetes.

Rebuttals to Negative Testimonials

Supposedly the people Isagenix hasn't helped haven't been following the program correctly. Apparent bad reactions are just signs that it is working: “When one is cleansing out years of accumulation of toxins, chemicals, jet fuel, gasoline, arsenic, heavy metals, radiation poisoning-one will have reactions.”

‘Evidence' That It Works

One commenter heard a doctor speak who cited all kinds of studies to support the theory behind Isagenix-that Isagenix cleansing can supposedly solve the problems of environmental toxicity, depletion of nutrients in the food supply, gastrointestinal malabsorption, and our incessant food cravings.

Here are some of the other commenters' opinions, a few of which I've replied to in brackets.

A former Hare Krishna was impressed by the array of nutrients in the products and believed that the doctor on the website had integrity and cared about her patients.

Several people claimed that we need nutritional supplements because the ground has been depleted of nutrients.

“There have been many valid scientific research [sic] to back the claims of Isagenix.” [I couldn't find any, and they provided no clues as to where to look.]

Others claimed that because lots of MDs are recommending Isagenix it must work; these MDs can't all be quacks. [Apparently they can. Lots of MDs recommend homeopathy, and some of them believe in astrology.]

Some commenters pointed out that Isagenix has paid for independent studies. [Where are they? What did they show? If Isagenix was paying, were they truly independent?]

Mainstream physicians are starting to realize cleansing is important, other commenters claimed. [Not any of the ones who practice science-based medicine.]

One commenter proposed that cleansing makes sense because one of the main ingredients of pesticides and insecticides is estrogen. It makes women fat and casues erectile dysfunction in men. Toxicity is a bigger cause of obesity than most people realize.

Another commenter insisted that because these products are “designed and formulated by professionals and advocated by professionals,” they must work.

One MD commenter claimed, “I have the before and after pictures and the lab tests to prove it.”

Pseudoscientific claims peppered many comments, such as this one: “Most people only absorb 8% to 12% of what we eat-the rest is waste which we flush down the toilet. With Isagenix we can absorb up to 94% of what is ingested with less waste going down the toilet. Isagenix is full of good probiotics which help rebuild our digestive systems, fights candida. Isagenix also helps the body become alkaline, which is a healthy body. John Hopkins 2008 Cancer Report stated that cancer cannot live in an alkaline body only acidic bodies. Processed food makes our bodies acidic-thus the epedemic [sic] of cancer and diabites [sic] in the USA along with heart disease.” [This is all nonsense.]

Isagenix is food, many commenters insisted. Regular food is from depleted soils. Organic food made children behave better at lunch in a school study. Genetically modified food is lacking in nutrition. “The majority of people fill their stomachs with foods void of natural nutrition and the evidence supports that they behave poorly, learn less, misbehave more and commit more crimes than those who fill their stomachs with highly nutritious organic produce and meats.” [Wow! Instead of the Twinkie defense, criminals can claim their non-organic lunch made them do it!]

“Isagenix is a divine blessing in this toxic sick world.”

These people apparently expect us to believe unsubstantiated assertions.They have no concept of what constitutes scientific evidence or why controlled studies are needed.

Defense of Multilevel Marketing

“MLM is not a scam, but one of the last bastions of free enterprise.” Some commenters claimed that MLM is good because products approved by the Food and Drug Administration (FDA) don't work, and that MLM is “the most legitimate business out in the world today.” All corporations are a pyramid anyway, they said.

But one commenter called MLM an “exploitative business model” and pointed out that the average yearly income for Isagenix distributors is only $116.87. Another pointed out that 97 percent of MLM schemes fail.

Personal Attacks on Me

“A Dr Harriet Hall wrote a very funny one sided arguement [sic] against [Isagenix] but omitted to inform the world how much money she has made conning patients into taking drugs she should know are harmful to you.”

Some commenters thought I was arrogant: “If it were up to know-it-all MDs like Harriet Hall, I'd still be in chronic pain.”

“To [sic] bad when you look up Dr. Hall in Washington no such person is licensed to practice medicine. Sad day when you have to lie to get people to pay attention to anything you say....” [It took me about one minute to locate verification of my license at https://fortress.wa.gov/doh/providercredentialsearch/SearchResult.aspx.

One commenter questioned whether I am really a doctor and says I have a small brain and a big mouth.

One claimed I write only to feed my ego.

Another said I shouldn't make comments without doing any research.

One thought I should try it for myself.

Another questioned why I didn't learn more by attending a meeting for the product, interviewing company representatives, or talking to the press.

Some thought instead of writing for the public I should have contacted the doctors at the company and discussed my concerns with them.

“Don't try to convince us, Dr. Hall, that you necessarily have ‘the answer.'” [Did I say I did?]

One alleged that I came to a conclusion without any research whatsoever; this is from a doctor who said, “Cleansing is now my first choice for my patients.” One wondered what research he did to make that choice.

“Going out of her way to trash Isagenix this way is pathetic.”

“PS ‘Dr.Hall' your little family practice designation really doesnt buy alot [sic] of cred.”

“Real doctors don't waste their time sitting on the internet making bogus posts about different health products.... could sign as doctor and no one would know.”

“This article is and the author is full of crap. I know it and he knows it.” [I know I'm not a “he.”]

Some commenters thought I didn't know anything and I should just shut up.

“This is just another doctor that stands to loose [sic] their income by the masses becoming healthy.”

“What ever [sic] Dr. Harriet Hall is selling, I'm not interested.” [For the record, I'm retired and the only thing I'm “selling” is critical thinking.]

Some suggested that just because I went to medical school doesn't mean I'm a smart person.

Kudos

A few commenters offered agreement and praise; they pointed out that no one had actually addressed any of the points I made or offered any evidence that what I wrote was wrong. They reprimanded other commenters for resorting to ad hominem attacks.

Attacks on the Medical Profession

Many of the commenters seemed to think that doctors know nothing about nutrition. Doctors just put bandages on problems: they sell pills that mask symptoms and wreak havoc on your body instead of treating underlying causes. They only want to make money. They want to keep people sick so they won't lose their kickbacks. [What kickbacks?] There are lots of malpractice suits.

“Most MD's [sic] will not even take the death dealing treatments they inflict upon the rest of the population.”

Some commenters claimed that even if evidence showed Isagenix worked, conventional medicine still wouldn't adopt it because of competition from drug companies. Many doctors are typically overweight and/or out of shape. The majority of emergency department doctors are lacking skills in emergency procedures.

One person commented, “MD's [sic] keep American's [sic] addicted to drugs! MD's also fancy themselves as God like. They think that being an MD allows them to keep American's from seeking nutrition.”

“Our medical doctors have failed us,” one person lamented.

Another observed: “So sad that people in our medical profession have no idea what they are talking about!!!”

Attacks on Science

Commenters insisted that instead of listening to science, one should listen to one's own body.

Some asked: even if it's only a placebo, why not use it?

Western medicine is trying to “squash Eastern medicine,” one commenter believed.

Another warned: “Things work for different people. Chiropractic and acupuncture work. If you ask for everything to be backed by studies, they just tailor the studies to benefit industry. Research things for yourself and don't be a sheep taking pills from an MD.”

Two commenters attacked the scientifically impeccable website Quackwatch, asserting that Stephen Barrett is literally funded by Big Pharma, the American Medical Association (AMA), and the FDA to produce disinformation aimed at discrediting alternative medicine. [He has no ties to any of those organizations.]

“See how herbs can treat people, not drugs,” one commenter advised.

“Did any of you see Sicko? If you did how could you possibly take one physicians [sic] ‘opinion' about something she didn't even try over the many testimonials.”

Some commenters felt they knew better than any doctor: “I choose to observe how my own body feels and reacts to what I ingest.”

“If you think its [sic] going to help it will,” one commenter suggested.

Some put forth that the real answer is to integrate Eastern with Western medicine.

“Oh, and I have found prayer helps me,” one Isagenix proponent added.

One commenter tried to turn the tables on me: “I feel it is unfair to say Isagenix is making unsubstantiated claims, and that it doesn't actually help you at all....... isn't that an unsubstantiated claim too?” [I didn't claim that it didn't work; I said there was no evidence that it did, and no reason to think it would.]

Attacks on the FDA and Big Pharma

Many commenters suggested that the FDA disclaimer about Isagenix is meaningless and believe we shouldn't take FDA warnings seriously: “It is a terrorist organization that lies, cheats steals, and intimidates anyone who stands between them and the targets of their wrath.”

“Dr Hall if you think the FDA is doing a good job you must love some of the poison they approve, such as Aspartame.”

Some commenters erroneously thought doctors got commissions for prescribing drugs.

One even asserted that a conspiracy of J.D. Rockefeller is behind the pharmaceutical industry and that many prescriptions are made from manipulation of petroleum.

People die from drugs, commenters insisted.

“My doctor wanted me to start beta blockers, after much investigation I decided that I was to [sic] young to have my liver contaminated by these pills... .”

Many commenters assured us that natural remedies work just as well and are safer than prescriptions.

Several commenters fervently believed that pharmaceuticals are the ultimate money-making scam.

Off-the-Wall False Claims

“The FDA (yes, those great friends of ours) just recently put a new advisement out there that we will soon be required to irradiate ALL raw vegetables and fruits [it certainly did not!]. Do you all know what irradiation does to food? It not only kills ‘bad' things like e. coli, but it kills nutrients from your foods as well.”

Try It for Yourself

Numerous commenters seemed to think the best way to determine if a treatment works is to try it yourself. But one commenter rightfully pointed out that the try-it-yourself argument is fallacious and condescending: “One does not have to experience snake venom to know to stay away from snakes.”

Haven't Tried It But Plan To

Several commenters were planning to try it after reading the article and comments. One of these said he knows firefighters who use it and he “would rather have one of the firefighters doing brain surgery on me, than let the average physician tell me what is going on in my body.” [Wow! Does this guy even have a brain?]

It's a Scam

Quite a few people agreed with what I wrote. Several were outspoken in calling Isagenix a scam.

“People would rather rave about this crap than admit that they were fooled into wasting their money.”

“Without even considering the science, common sense helped me spot this as bullshit.”

“Isagenix is a freakish cult perpetrated on the uncritical, by the unscrupulous, using the desperate search for the ever-elusive ‘easy solution.'”

One reported that a cousin and her boyfriend are “making a TON of money selling this stuff to all of you morons stupid enough to buy it and make them rich. ISAGENIX only ‘works' for the people selling it. Diet and exercise WORKS for everyone!”

Concerns

A few commenters expressed concerns about the product. One commenter said the Isagenix company representative couldn't answer questions about origin of ingredients and quality control. There have been no controlled studies. Where is the evidence? How do we know it is safe? Long-term results remain to be seen. How many can maintain this restrictive lifestyle for years? Why isn't Isagenix being regulated by the FDA? “I am a little concerned about the way some people discuss this product in almost cult-like fashion. It makes me wonder if there are mind-control drugs in this stuff.”

Two Jokes

“I got a refund check from [the] IRS after starting Isagenix.”

“I have some magic beans for sale. Try eating right and exercising instead.”

Funny, Unhelpful, and Bizarre Comments

“Who cares whether it works or not. This stuff tastes like 9-day old garbage mixed with water from a sewer.”

One man took it on the recommendation of his chiropractor; he now distrusts both Isagenix and his chiropractor. “I have been feeling better ever since I stopped having my head wrenched and being put on a rack and practically decapitated week after week, except for the apparently permanent click in my neck that wasn't there before.”

“We fertilize our soil with fake nutrients and usually do not replace with all 60 nutrients the plants need to be healthy so they are prone to diesease [sic-a disease that they die from?] and incests [sic].” [Gotta watch out for those incestuous plants!]

“I never hear anything from the medical field about elevating the PH level in the human body to keep in from being to acidic. That study was done by Dr Lioness Paulings medical reseacher and nobel prize winner.” [Errors in original. Lioness?!]

“Whoever started this blog is an idiot.”

“I am amazed at the amount of ingnorance [sic] on this Blog. Whom [sic] ever allows this should be ashamed.”

My favorite comment of all was “Dr Harriet Hall is a refrigerator with a head.” I don't know what that means, but its whimsical imagery appeals to my sense of humor.

In looking back at this whole kerfuffle, it became clear to me that there had been a colossal barrier to communication. The person who had originally asked me about Isagenix, the blog owner, and I were all operating in the arena of science and evidence. Most of the commenters were operating in a whole different universe of discourse based on belief, hope, hearsay, and personal experience. Science is like a foreign language to them, and they were incapable of understanding my points. Pearls before swine... ]]> Thu, 06 Jan 2011 22:20:00 EDT info@csicop.org () http://www.csicop.org/si/show/a_gifted_writer_and_a_book_worth_giving http://www.csicop.org/si/show/a_gifted_writer_and_a_book_worth_giving Evolution: How We and All Living Things Came to Be.
By Daniel Loxton. Kids Can Press, Toronto, 2010.
ISBN: 978-1554534302. Hardcover, $18.95.

It’s hard to believe that we still have so many evolution deniers among us. Understanding evolution is essential to understanding modern biology as well as a host of other subjects. We need to get to young minds before their neurons have a chance to congeal into unscientific ideologies. Now we have just the book to reach them.

Daniel Loxton is the editor of the “Junior Skeptic” section of Skeptic magazine, where he makes skepticism and critical thinking accessible and entertaining to the younger set. He has expanded one of his “Junior Skeptic” subjects into a superb new book on evolution.

The illustrations are colorful, informative, and whimsical. Loxton introduces us to a blue bird that compromises on a tail that is “not too long, not too short,” some cute “Zooks” that move away and eventually lose the interest and ability to mate with the others, a boy overrun with bunnies that have reproduced without anything to limit survival of the offspring, and some really cool dinosaurs. They’re a joy to the eye, and the text is a joy to the mind.

Loxton covers the basics of evolutionary theory; tells the story of Darwin, the Beagle, and the finches; and answers the questions people commonly ask:

“But have we ever actually seen a new species evolve?” Yes, both in the wild and in the lab.

“Where are the transitional fossils?” Everywhere.

“Didn’t they find some human footprints together with dinosaur footprints?” No, they made a mistake.

“How could evolution produce something as complicated as my eyes?” Loxton shows us how complex eyes gradually developed from simple light-sensitive cells.

“How could walking animals turn into flying animals?” Perhaps from gradual alterations in tree-dwelling, gliding animals.

Then the hard questions: “How did life start in the first place?” Evolution doesn’t explain the origin of life, just how it changed over time. We don’t know how life got started, but scientists are working on it.

And “What about religion?” Loxton handles this neatly by saying that this is a question science can’t help with. He refers readers to their “family, friends, and community leaders.” (He avoids mentioning rabbis, imams, priests, or Flying Spaghetti Monsterologists.)

Loxton has a wonderful knack for simplifying without condescending and for challenging young readers to grapple with complicated concepts. The book is aimed at eight–to thirteen-year-olds, but it could be useful to even a sophisticated old coot. Some of his examples might come in handy in your next discussion with an intelligent design believer or a fence sitter. I loved his illustration of how evolutionary change is not a totally random process but builds on patterns that were already there. He describes how hot-rod builders can lift a car, drop it, chop the roof, and slap on new paint, but they are still stuck with the basic pattern of a body and four wheels (not two or seventeen).

If you have children or grandchildren, this book would be a great way to introduce them to the theory of evolution. If you don’t, you still might want to buy a copy, read it yourself, and donate it to the local public or school library.

I hope Loxton will write many more books like this on a wide variety of skeptical subjects. He has a gift, and we are fortunate that he is sharing it with us.

Carrying a Power Balance card in your pocket will supposedly improve your athletic performance and cure what ails you. The alleged mechanism ("frequencies" in an embedded hologram) is laughable pseudoscientific bunk.

Remember when professional golfers were wearing Q-ray bracelets to improve their game? The Q-ray folks recently had a run-in with the courts. They admitted their product was only a placebo but argued that it was acceptable to lie to elicit the placebo response. The judge disagreed: they were convicted of fraud, forced to pay back $16 million, and required to remove the deceptive claims from their advertising. Now they have a new competitor: Power Balance Performance Technology. Like the Q-ray bracelet, it is based on "resonance." It doesn't even have to come in contact with your body: one version is a card that you simply put in your pocket.

Power Balance representatives demonstrate their products

in sports stores at malls. They test your strength and balance and then give you a Power Balance card to hold or put in your pocket. When they retest you, you miraculously do better. There are some revealing videos on YouTube, including a short clip that shows the subject standing on one foot with arms outstretched. The salesman pushes down on the subject's arm near the wrist, and the subject starts to fall over. After the subject puts a Power Balance card in his pocket, the salesman repeats the test but this time pushes down near the elbow, creating a shorter lever arm that of course reduces the effect of the force applied, so the subject doesn't fall over. In other demonstrations, they use other simple biomechanical tricks like this to create false impressions of improved strength. The amount of force applied is subjective, both parties know when the card is in use, and they know what is expected to happen—it's a recipe for self-deception.

What's in these magic cards? I will quote at length from their Web site for the entertainment value:

POWER BALANCE Performance Technology has been embedded with naturally occurring frequencies found in nature that have been known to react positively with the body's energy field. This helps to promote balance, flexibility, strength and overall wellness.

For thousands of years, eastern medicine has been using the same techniques for personal wellness through finding things in nature that react positively with your body, such as rocks, minerals, crystals, etc. Through kinesiology we have learned that certain foods cause the body to react either positively or negatively as well. Although not all substances found in nature work the same on everyone, we have narrowed it down to a few that we believe are highly beneficial and have put them together to create Power Balance Performance Technology.

It's hard to argue with nature and the fact is that everything in nature resonates at a particular frequency. That is what keeps it all together. We react with frequency because we are a frequency. Most simply, we are a bunch of cells held together by frequency. If you hold processed sugar or a cell phone in your hand and hold your arm straight out to your side and have someone push your arm down while you resist, it goes down pretty easily because processed sugar and cellular telephones do not react positively with the human body. Basically, the frequencies in sugar and cell phones create a reaction that makes your body weaker. Adversely, if you put certain vitamins or minerals in your hand and do the same test with your arm, you will find it is much harder for that person to push your arm down. Your body's energy field likes things that are good for it and craves to be around those things. At Power Balance, we have taken a few of those items and through advances in technology, have been able to duplicate those positive energies and imprint them onto our holographic media.

Why Holograms? We use holograms because they are composed of Mylar—a polyester film used for imprinting music, movies, pictures, and other data. Thus, it was a natural fit. In fact, the hologram is so complex with such infinite depth and minimal surface area, that many companies are now using them as hard drives. Along those same lines, we felt that it would be a lot easier to get someone to put a hologram in there [sic] shoe rather then [sic] a Power Balance equipped rock or apple.

Power Balance products include a ten-pack of stick-on embedded holograms ($59.95), a pendant ($39.95), a wristband ($29.95), and an eight-pack of pocket cards ($59.95).

The company targets athletes, particularly surfers. According to numerous testimonials, Power Balance seems to improve performance. One surfer claims he can even sense the presence of the card: "I can feel it on me." Another testimonial is from Tommy Grunt, United States Marine Corps. Maybe Grunt is real, but ads for quack products have been known to feature fabricated testimonials, and I can easily imagine a copywriter putting tongue in cheek and creating a name like that to relieve the boredom. There are reports of the products' effectiveness in animals, from horses to birds. The products allegedly relieve headaches, menstrual pain, and all kinds of other symptoms. The testimonials give the impression that if you feel unwell in any way, the magic card will restore you to normal. If you already feel well, it will make you better than normal.

"A primitive form of this technology was discovered when someone, somewhere along the line, picked up a rock and felt something that reacted positively with his body." I don't doubt that someone believed he felt something, but I seriously doubt it was due to the frequency of the rock resonating with the frequency of his body.

For resonance to occur, something has to vibrate. You may be able to make a rock resonate, but the rock doesn't create its own vibrations. Crystalline structures can be made to vibrate. The tympanic membrane and the vocal cords vibrate, but the whole body doesn't. When a soprano wants to break a glass with her voice, she can first listen to the sound made by tapping it with a spoon; if she can match that sound frequency, the glass will resonate and possibly shatter. How can you tap a cat to see what its frequency is? Can you imagine a soprano shattering a cat?

This whole resonance and vibration business is pseudoscience emanating from the myth of the human energy field—not the kind of energy physicists measure but some vague life energy like the acupuncturists' qi, the chiropractors' Innate, and the imaginary fields that Therapeutic Touch practitioners claim they are smoothing down with their hands. "We are a frequency" and "We are a bunch of cells held together by frequency" and "Your body's energy field likes things that are good for it" are statements so incoherent, so much at odds with scientific knowledge, that they "aren't even wrong."

The definition of frequency is "the number of repetitions of a periodic process in a unit of time." A frequency can't exist in isolation. There has to be a periodic process, like a sound wave, a radio wave, a clock pendulum, or a train passing by at the rate of x boxcars per minute. The phrase "33 1/3 per minute" is meaningless: you can't have an rpm without an r. A periodic process can have a frequency, but an armadillo and a tomato can't. Neither a periodic process nor a person can "be" a frequency.

Pushing down on the arm is a bogus muscle testing technique known as applied kinesiology. It is supposedly used to diagnose allergies: if you hold a sealed vial of an allergen, your strength supposedly diminishes. It only works if the doctor and patient know what substance is being tested; when double-blind controls have been used, kinesiology has failed every test.

Omitting for a moment the crucial question "Frequencies of what?" how did the Power Balance creators determine which frequencies to use? "We have narrowed it down to a few that we believe are highly beneficial." Okay ... how exactly did they measure the frequencies, and what criteria did they use to narrow them down? I think the wording of the ad is revealing: the company says they "believe" they are highly beneficial, not that they have any evidence that they are—assuming there really are any frequencies and that they have somehow put them in a hologram. I e-mailed the company and asked simple questions like "How do you measure the frequency of a rock?" They didn't answer.

In online discussions, one man "tested" the product by having one hundred athletes try it, with no controls of any kind; not surprisingly, all of the athletes reported improvement. A man watching a demonstration suggested a real test, blinding the subject as to whether the card was present, but (not surprisingly) the salesman wouldn't cooperate.

This would be so simple to test properly. Take five Power Balance cards and five credit cards, put them in opaque envelopes, shuffle, number the envelopes 1 through 10, have a third party slip an envelope in the subject's pocket, and then challenge the salesman to tell which envelopes had the real card. I could not find evidence that they have ever done such a test, presumably because they know it would fail.

These products may actually do some good. Modern versions of an amulet or rabbit's foot (without harm to rabbits), they elicit a placebo response, giving people confidence and possibly making them try harder. They are not exorbitantly expensive and even come with a money-back guarantee.

The marketing is pure genius. If I were a professional scam artist, I don't think I could come up with anything better. The company has an impressive trick demonstration that easily fools most people. They spout a lot of pseudoscientific hooey that sounds impressive to the scientifically illiterate, but they are careful to make only vague claims that the Federal Trade Commission can't object to. The harmless products are inexpensive to manufacture, but the company charges enough to afford a money-back guarantee and still make money. They package the cheaper cards and stickers in multiples so they can charge more, but the prices are still low enough that the average person is willing to take a chance. Who knows what is actually in the products? If it were my scam, I'd put in any old hologram or none at all. No one is likely to investigate your production line to see how you get all those "beneficial frequencies" into the Mylar.

Tell me you use the Power Balance card and it makes you feel better, and I can readily believe you. Tell me your performance improves when you carry it, and I will believe you. But that won't convince me that the improvement has anything to do with bioresonating frequencies in the holograms—or even with the cards themselves.

It's like the tooth fairy. Tell me money appears under your pillow, and I will believe you. But that won't convince me that the tooth fairy did it.

The tooth fairy phenomenon is easily explained by human psychology and parental behavior. The Power Balance phenomenon is easily explained by suggestion, confirmation bias, the placebo response, and other well-known aspects of human psychology that conspire to persuade people that ineffective things work.

Before writing this article, I discussed with CSI Research Fellow Benjamin Radford whether the Committee for Skeptical Inquiry might want to do a simple double-blind test. We decided not to because it is just too silly to bother with. As Radford put it, "This sort of scientific testing should be done by the company; it is not the skeptics' job to spend time and money testing outlandish claims for which no reliable evidence has been offered."

We're not going to bother setting up a video camera to catch the tooth fairy either.

Chiropractors, homeopaths, naturopaths, acupuncturists, and other alternative medicine practitioners constantly criticize conventional medicine for “only treating the symptoms,” while alternative medicine allegedly treats “the underlying causes” of disease.

Nope. Not true. Exactly backwards. Think about it: When you go to a doctor with a fever, does he just treat the symptom? No, he tries to figure out what’s causing the fever. If it’s pneumonia, he identifies which microbe is responsible and gives you the right drugs to treat that particular infection. If you have abdominal pain, does the doctor just give you narcotics to treat the symptom of pain? No, he tries to figure out what’s causing the pain. If he determines you have acute appendicitis, he operates to remove your appendix.

I guess what they’re trying to say is that something must have been wrong in the first place to allow the disease to develop. But they don’t have any better insight into what that something might be than scientific medicine. All they have is wild, imaginative guesses. And they all disagree with one another. The chiropractor says that if your spine is in proper alignment, you can’t get sick. Acupuncturists talk about the proper flow of qi through the meridians. Energy medicine practitioners talk about disturbances in energy fields. Nutrition faddists claim that people who eat right won’t get sick. None of them can produce any evidence to support these claims. No alternative medicine has been scientifically shown to prevent disease or cure it. If it had, it would have been incorporated into conventional medicine and would no longer be “alternative.”

Are these practitioners treating the underlying cause, or are they simply applying their one chosen tool to treat everything? Chiropractors treat every patient with chiropractic adjustments. What if a doctor used one treatment for everything? You have pneumonia? Here’s some penicillin. You have a broken leg? Here’s some penicillin. You have diabetes? Here’s some penicillin. Acupuncturists only know to stick needles in people. Homeopaths only know to give out ridiculously high dilutions that amount to nothing but water. Therapeutic touch practitioners only know to smooth out the wrinkles in imaginary energy fields. They are not trying to determine any underlying cause; they are just using one treatment indiscriminately.

How do you define “cause”? We don’t know what causes gravity, but we understand enough about how it works to overcome it with elevators, airplanes, and rockets to the moon. We may not know what ultimately causes asthma, but we know enough about the causes of airway constriction and inflammation to devise effective treatments.

Let’s take a simple example: strep throat. The symptom is throat pain. Doctors don’t just treat the pain; they do a throat culture, they determine that a strep infection is causing the pain, and they treat the infection with an antibiotic. But what caused the strep infection? The body had to host the bacteria and respond to their presence by developing symptoms; the bacteria had to be capable of multiplying in the human body. The patient had to be exposed to another person who had a strep infection, who in turn had caught it from someone else, involving a chain of social and epidemiologic causes. The bacteria had to evolve from ancestor bacteria and the human from ancestor animals. And so on.

So you see, it involves a chain of causation and there can even be several simultaneous causes. “Cause” can mean pretty much anything you want it to. But however you look at it, doctors definitely do not “just treat symptoms.”

Philosophy has studied causation. Aristotle said everything had four causes: material, formal, efficient, and final. And he introduced complications: proper (prior) causation and accidental (chance) causation. Potential or actual, particular or generic. Reciprocal or circular causality as a relation of mutual dependence or influence of cause upon effect. The same thing as the cause of contrary effects when its presence and absence result in different outcomes. He recognized that the subject of causation was complicated.

Alternative providers are more “simple” minded. They often claim to know the one true cause of all disease, which is curious because medical science defines several categories of causes falling under the mnemonic VINDICATE:

V – Vascular

I – Infectious/inflammatory

N – Neoplastic

D – Drugs/toxins

I – Intervention/iatrogenic

C – Congenital/developmental

A – Autoimmune

T – Trauma

E – Endocrine/metabolic

And sometimes more than one cause is involved (e.g., a traumatic injury gets infected). Where science finds complexity, alternative medicine imagines simplicity. As H.L. Mencken said, “For every complex problem, there is an answer that is clear, simple—and wrong.”

Some homeopaths claim to treat “genetic” illness, tracing its origins to six main genetic causes: tuberculosis, syphilis, gonorrhea, psora (scabies), cancer, and leprosy. Bet you didn’t know tuberculosis was genetic! Neither did I. Science classifies all these as infectious except for cancer, which is neoplastic. Homeopathy disregards science and redefines genetic to suit its own inscrutable purposes.

Science finds many causes for disease and sometimes more than one cause for a given disease. Pseudoscience has identified the one true cause of all disease—many times. I did an Internet search and found sixty-seven single causes of all disease (see accompanying box). This is not an exhaustive list but rather an exhausted list (I stopped when I got tired of searching).

It never seems to bother proponents of alternative medicine that others have found different “one true” causes. In his book Voodoo Science, Bob Park describes a press conference following a meeting to discuss government funding for alternative medicine research:

Perhaps the strangest part of the press conference consisted of brief statements by individual members of the editorial review board of what they saw as the most important issues for the Office of Alternative Medicine. One insisted that the number-one health problem in the United States is magnesium deficiency; another was convinced that the expanded use of acupuncture could revolutionize medicine; and so it went around the table, with each touting his or her preferred therapy. But there was no sense of conflict or rivalry. As each spoke, the others would nod in agreement. The purpose of the OAM, I began to realize, was to demonstrate that these disparate therapies all work. It was my first glimpse of what holds alternative medicine together: there is no internal dissent in a community that feels itself besieged from the outside.

When scientists encounter two mutually exclusive claims, it bothers them. They experience cognitive dissonance and try diligently to find evidence to reject one of the hypotheses and leave a winner. They eventually reach a consensus. Alternative medicine pseudoscientists don’t seem to mind cognitive dissonance. They are content to look for evidence to support their own chosen treatment while blithely disregarding competing claims. They don’t want to look for evidence that something doesn’t work. While each claims to know the one cause of disease, they don’t seem interested in looking for the one truth.

Live and let live? Create your own reality? Truth is only relative? The same thing may be simultaneously true for me and false for you? Maybe it boils down to a mutual tolerance of delusions (okay, I’ll believe that you are Jesus if you believe that I’m Napoleon). For the cynical, follow the money: “I won’t interfere with your livelihood if you don’t interfere with mine.”

I can play the cause-finding game too. I’ve discovered the one cause of all the one-cause theories: a deficiency of critical-thinking skills combined with an overactive imagination. And, of course, a failure to test beliefs using the scientific method.

I recently read Flock of Dodos: Behind Modern Creationism, Intelligent Design, and the Easter Bunny (Barrett Brown and Jon P. Alston, Cambridge House Press, New York, 2007, no relation to the movie Flock of Dodos). It’s a hilarious, no-holds-barred send-up of the lies and poor reasoning employed by the intelligent design movement. I was particularly struck by a quotation from William Dembski’s book Intelligent Design: “We are dealing here with something more than a straightforward determination of scientific facts or confirmation of scientific theories. Rather we are dealing with competing world-views and incompatible metaphysical systems.”

That doesn’t just apply to intelligent design. It cuts to the essence of what skeptics encounter on every front, from

dowsing to homeopathy, from ESP to therapeutic touch. We are trying to evaluate the science behind claims that are often not based on science but on beliefs that are incompatible with science. The claimants are happy to use science when it supports them, but when it doesn’t they are likely to unfairly critique the science or even to dismiss the entire scientific enterprise as a “materialistic worldview” or “closed-minded.” We are talking at cross purposes. How can we communicate if we say “this variety of apple is red,” and they insist “it feels green to me”?

We get frustrated when we show these folks the scientific evidence and they refuse to accept it. Dowsing fails all tests, but dowsers “know” from personal experience that it works for them. Homeopathy is not only implausible, but it has been tested and has failed the tests. Yet proponents refuse to acknowledge those failures and still want to talk about data from the nineteenth century and make claims for the memory of water. We have to realize we are not even speaking the same language. We are trying to play a civilized game of gin rummy, and they are dribbling a basketball all over the card table. Before competing, doesn’t it make sense to define what game you’re playing and what the rules are?

Before arguing with a mathematician about the solution to a geometry problem, it’s essential to establish whether he is following the rules of Euclidean geometry, where parallel lines never cross, or non-Euclidean geometry, where they sometimes do.

Science has been a very successful self-correcting group endeavor. It wouldn’t be successful if it didn’t follow a strict set of rules designed to avoid errors. (Note: there are no rules written in stone; I’m talking about conventions that are generally understood and accepted by scientists, conventions that grow naturally out of reason and critical thinking.) If proponents of intelligent design or alternative medicine want to play the science game, they ought to play by the rules. If they won’t play by the rules, they effectively take themselves out of the scientific arena and into the metaphysical arena. In that case, it is useless for us to talk to them about science.

If you want to play the science game, here’s what you do:

1. Submit your hypothesis to proper testing. Testimonials, intuitions, personal experience, and “other ways of knowing” don’t count.
2. See if you can falsify the hypothesis.
3. Try to rule out alternative explanations and confounding factors.
4. Report your findings in journal articles submitted to peer review.
5. Allow the scientific community to critique the published evidence and engage in dialogue and debate.
6. Withhold judgment until your results can be replicated elsewhere.
7. Respect the consensus of the majority of the scientific community as to whether your hypothesis is probably true or false (always allowing for revision based on further evidence).
8. Be willing to follow the evidence and admit you are wrong if that’s what the evidence says.

If you want to play the science game, here are some of the things you don’t do:

1. Accuse the entire scientific community of being wrong (unless you have compelling evidence, in which case you should argue for it in the scientific journals and at professional meetings, not in the media).
2. Design poor-quality experiments that are almost guaranteed to show your hypothesis is true whether it really is or not. Use science to show that your treatment works, not to ask if it works.
3. Keep using arguments that have been thoroughly discredited. (The intelligent design folks are still claiming the eye could not have evolved because it is irreducibly complex; homeopaths are still claiming homeopathy cured more patients than conventional medicine during nineteenth-century epidemics).
4. Write books for the general public to promote your thesis—as if public opinion could influence science!
5. Form an activist organization to promote your beliefs.
6. Step outside the scientific paradigm and appeal to intuition and belief.
7. Mention the persecution of Galileo and compare yourself to him.
8. Invent a conspiracy theory (Big Pharma is suppressing the truth!).
9. Claim to be a lone genius who knows more than all scientists put together.
10. Offer a treatment to the public after only the most preliminary studies have been conducted.
11. Set up a Web site to sell products that are not backed by good evidence.
12. Refuse to admit when your hypothesis is proven wrong.

Changing Our Minds

Scientists will change their minds when the evidence warrants. Before we waste time arguing, one thing we can do is ask our opponents what it would take to change their minds. One woman I asked said no amount of evidence could change her mind because she knew from personal experience that her claim was true, so any evidence that said otherwise would have to be false and fabricated. End of discussion. She’s out of the game.

The rules of science are pretty clear about what it takes to change our minds. I’ll use the example of Helicobacter and ulcers. We used to think that stress and too much stomach acid caused ulcers; now we think a bacterium causes ulcers. Here’s a summary of why we changed our minds:

1. Scientists noticed bacteria in biopsy samples from ulcers.
2. They identified the bacteria as Helicobacter pylori.
3. They found a strong correlation between ulcers and the presence of the bacteria.
4. One of the researchers, who was healthy and not a Helicobacter carrier, was able to induce an ulcer in himself by ingesting the bacteria.
5. They found that treating patients with antibiotics cured ulcers.
6. They found that antibiotics were superior to previous ulcer treatments.
7. The studies were replicated and conducted in different ways that corroborated each other.
8. The bacterial hypothesis was not inconsistent with the rest of scientific knowledge.

If we had the same quantity and quality of evidence for homeopathy, we’d gladly accept it. In fact, if the evidence met criteria 1 through 7, we’d provisionally accept it while we kept checking the data and tried like crazy to figure out the mechanism behind homeopathy. (For more on this, see “Bacteria, Ulcers, and Ostracism” in the November/December 2004 Skeptical Inquirer.)

There are two issues that are often misunderstood: scientific consensus and prior plausibility.

Prior Plausibility

Homeopathy is completely implausible. We would have to accept robust evidence that it worked, but we would require much stronger evidence than we would for, say, a new antibiotic. If the claims for homeopathy were true, we would have to revise much of what we know about physics, chemistry, and physiology.

The crossword analogy is helpful. If you think the answer to 1-across should be “library” but the clue to 1-down is a five-letter word for the author of Tom Sawyer and the clue to 2-down is a four-letter-word for the name of Eve’s husband in Genesis, you have to reject “library” and keep looking for a word that starts with T-A. You have to recognize that no matter how strong your conviction that 1-across must be “library,” you must be wrong and there must be another answer that you just haven’t considered.

Consensus

It’s easy to dismiss the scientific consensus as a popularity contest, a vote on opinions. But it’s far more than that. The body of evidence stands or falls on its own merits, and when the weight clearly tips the balance to one side, everybody can see it. The scientific community is made up of experts who know how to evaluate the evidence and who thrash out disagreements in medical journals and scientific conferences. It is easy for the scientific community to reach an agreement based on clear evidence. There are times when the evidence is less clear and controversy among scientists is appropriate, but there comes a time when it would be perverse not to accept the evidence, just as it is perverse to deny evolution or germ theory. The scientific consensus on evolution and the germ theory is a recognition of reality, not a matter of opinion.

A reasonable default assumption is that the scientific consensus is usually right; if it isn’t, it will change as the evidence becomes clearer. Truth will prevail. It does no good to attack the scientific consensus as prejudiced or closed-minded. The consensus will change only when it incorporates new and better evidence. One of the irrational tactics we’ve seen over and over is for opponents to cite one or a handful of studies to support their belief. They ridiculously assume that it was new information that the people who reached the scientific consensus had failed to consider or that it somehow outweighs all the other studies that found the opposite to be true.

Play by the Rules or Go Play Your Own Game

There’s no point in arguing scientific facts with someone whose worldview is metaphysical and nonscientific. There’s no point in presenting geological age data to someone who “knows” the age of the Earth from the Bible. Before we get into a useless debate, maybe we should find out what game our opponents are really playing. If they are playing ping pong, it’s silly for us to bring a football to the table. It would be handy if we could get them to say up front what game they are really playing, but all too often they have deluded themselves into truly believing they are following the rules of science.

If they won’t play the science game by the rules, we are justified in crying “foul” and disqualifying them. Then they can go away somewhere else and play their own game by whatever rules they want, and we won’t be able to refute them. If they are relying on beliefs unsupported by evidence, let them say so. Wouldn’t it be refreshing to hear a homeopath say, “I believe homeopathy works based on my personal experience and on nonscientific evidence like testimonials, and I categorically reject the results of any scientific trial that fails to support my beliefs. Homeopathy cured my neighbor’s uncle’s cousin of cancer. Trust me. I’m a nice guy so you should believe whatever I tell you.”

If they’d say that up front, we wouldn’t waste any of our valuable time rehashing scientific evidence that they will just ignore. They would be out of the game, permanently. And patients would have a better basis for giving truly informed consent.

I like to read advertisements for quack remedies. I’ve come to suspect that “clinically proven” means “we gave it to three of our friends and got them to say it worked.” When the ads cite published medical studies, I like to track down and read those studies. I usually find that they have nothing whatsoever to do with the product in question and I get a lot of amusement from the pseudoscience and the testimonials.

I really hit the jackpot when I noticed an ad for a weight-loss product called Akavar 20/50. It made the usual claims: eat all you want and still lose weight. But it had the best advertising slogan ever: “We couldn’t say it in print if it wasn’t true!” I laughed out loud. Anyone can say anything in print until they get caught. These diet ads all say things that aren’t true, and the Federal Trade Commission can’t begin to catch them all.

The ad describes research results on Akavar as “staggering.” It claims to have published scientific research showing that twenty-three out of twenty-four patients using Akavar’s active ingredient lost weight and describes a controlled, randomized clinical trial of the actual product in which twenty-three out of twenty-four patients lost “a substantial amount of weight.” Two questions immediately came to mind: why were the numbers the same in both studies, and if a single active ingredient worked just as well, why was there any need to develop the Akavar formulation?

There was a toll-free number to call for further information. I called and asked where I could read the two studies they referred to. The man who answered was flummoxed: “No one’s ever asked me that before.” He had to go for help. Finally he came up with the names of two journals but no further information.

I searched PubMed for anything in either of those journals that might even remotely be considered studies of Akavar and couldn’t find anything. I wrote the company’s customer service representative and asked for more information. That led to the following surreal e-mail exchange over the next month and a half.

September 30: [Me] Your ad for Akavar describes a high rate of success in clinical studies. I’d like to read those studies for myself. I called your 800 number and the person who answered told me there were two studies published in the Journal of Human Nutrition and Dietetics and in the journal Medical Psychopharmacology. He was unable to give me the full citations, and I have searched PubMed and elsewhere and have been unable to locate the articles. Could you give me the exact citations (date, author, title of article, journal, volume, and page number)? Or better yet, could you possibly send me electronic copies of the articles? I would really appreciate it.

October 9: [Akavar] Thank you for your interest in Akavar 20/50. I will be happy to submit a request to our Compliance dept. and have these studies prepared for you to send via email or via mail. We request to know as to what use these will be used for and will require a phone number and address. As soon as I have this information I will submit the request and have these prepared for you to save any trouble of having to look these up yourself. Thanks so much.

October 9: [Me] I would prefer you send them by e-mail. What they will be used for? To help me decide for myself whether there is adequate evidence to recommend Akavar 20/50 to patients.

October 10: [Akavar] Thank you so much for this. I will forward this request to compliance and send via email when they have finished preparing the study.

October 15: [Me] I’m still waiting. The delay is making me wonder... if you really have legitimate scientific studies to back up your claims, why are they not posted on your web site or linked to the PubMed abstracts or at least listed in such a way that they can be located by interested physicians?

October 17: [Akavar] I apologize for the delay. I will follow up with our Compliance/Legal department to see if they have prepared these for you or not. I will let you know shortly.

October 30: [Me] It is now October 30, and I still have not received the studies. If they are not available in electronic format, all I really need is a proper citation: title of article, name of journal, names of authors, date, volume and page. If these studies really exist, and if they really support your product, your company certainly doesn’t seem very proud of them! If you cannot provide me with the citations, I will be forced to assume they do not exist and I will report your company for false advertising.

November 2: [Me again] OK. Still no response. I will have to give you a deadline. If you have not sent me the citations by November 5, I will take it as an admission that you are crooks who tell deliberate lies in your advertising and I will report you to the FTC. I will remind you that ALL I’m asking is that you tell me where I can find the clinical studies you advertise as supporting your product.

November 2: [Akavar] I just spoke with our Legal department as I have been out of the office this week. They informed me that they are contacting you via mail as they are requesting more information from you. I can not handle this request other than our legal department. This was sent to the address you provided me below and should be received within normal postal delivery time. [I never received anything by mail.] I apologize sincerely for this delayed response. It should be taken care of now. Thank you.

November 2: [Me] How about you give me the e-mail address of the legal department so you don’t need to act as intermediary? There is no reason for them to request more information from me—that is ridiculous! And even if they are mailing me copies of the studies, there is no reason they can’t also immediately provide me with the citation information via e-mail. Reputable companies usually display that kind of information proudly on their web sites, often with a link to the studies.

November 5: [Akavar] Our compliance/legal department has prepared the following for you and are sending via email at your request via the above attachments. Please respond accordingly. Thanks again for your patience.

[Attachment] We have received your request to provide you with all studies relating to our Akavar 20/50 product. Due to the confidential nature of these studies, we cannot release these studies without a signed Non-Disclosure Agreement. Our standard Non-Disclosure Agreement is enclosed. Pleases [sic] review and sign the Agreement. Upon receipt of the signed Non-Disclosure Agreement, we will happily provide you the information you requested. . . . [This was accompanied by a complicated, multi-page legal document.]

November 5: [Me] You have GOT to be kidding!! I did NOT ask for “all” studies relating to your product. I did NOT ask for any proprietary information. All I asked for was the correct citations for the two published studies referred to in your advertising. This is not anything that requires any signature or agreement. Published studies are in the public domain. This is becoming a surreal experience. Perhaps I’d better start all over again by copying my initial request: [My initial e-mail was copied here.]

Let’s make this really simple:

1. Are there two published studies?
2. If so, please provide me with the information I will need to locate and read those studies: Name of author(s), title of article, name of journal, volume, page number and date of publication.

November 7: [Akavar] Any update from MKF?

November 7: [Me] No. Who or what is MKF?

November 13: [Akavar] We regret that you refused to sign the NDA, which would have allowed us to provide you the highly confidential, proprietary data related to Akavar. We are, however, enclosing the citations for the published articles relating to Akavar’s efficacy. [Lieberman, H.R., Tharion, W.J., Shukitt-Hale, B., Speckman, K.L., & Tulley, R. (2002). Psychopharmacology (Berl), 164(3), 250–261. Andersen, T. and J. Fogh (2001). J Hum Nutr Diet 14(3): 243–50.] Any representation on your part that the published studies comprise the full substantiation for Akavar 20/50 or that the substantiation is lacking in any way would be false and intentionally misleading on your part since your [sic] were not privy to the full documentation. Again because of your refusal to sign a simple NDA. [This letter was signed by a paralegal.]

November 13: [Me] You did not provide the titles of the studies, but I easily found them. I can see why you didn’t provide the titles, and I can see why I didn’t find them when I looked before, because it is obvious that they were not studies of Akavar 20/50.The Lieberman study is titled “Effects of caffeine, sleep loss, and stress on cognitive performance and mood during U.S. Navy SEAL training. Sea-Air-Land.” The Andersen study is “Weight loss and delayed gastric emptying following a South American herbal preparation in overweight patients.” The herbal preparation was a mixture of yerba mate, guarana, and damiana. The patients initially lost a few pounds, but those who took the active drug for 12 months “maintained” their weight during that period. The abstract of the study does not say that the study participants were instructed not to alter their eating habits. And the numbers of patients do not correspond to either of the studies described in your ads.

Your ad says, “this is scientific fact, documented by published medical findings.” Are you now admitting that there are no published clinical studies of Akavar 20/50 and that the statements in your ads are false?

I never heard back from them, and I decided I had had enough fun. I reported them for false advertising. I was not the only one to complain. A class action suit was filed against the company for “fraudulent, deceptive, and otherwise improper advertising and marketing practices.” The lawsuit says, “Akavar has not undergone scientific evaluation by a team of doctors, nor has Akavar been tested in controlled random clinical trials.” The lawsuit also mentions that Akavar is identical to another of the company’s products, Estrin-D, which is also the subject of an unrelated lawsuit.

Later a friend contacted the company and signed the nondisclosure agreement to see what would happen. All he got was a written summary of some unspecified studies with no authors or publications listed.

Imagine a pharmaceutical company telling me they couldn’t divulge the title of an article about their new drug in the New England Journal of Medicine unless I signed a nondisclosure agreement! What planet are we on? Even worse, imagine if a pharmaceutical company asked the FDA to approve a new drug on the basis of two studies that had little or nothing to do with that drug, insisting they had more proof, but it was a secret!

I don’t know why I’m surprised. Quacks have to defend themselves any way they can, since they can’t defend themselves with facts.

You might be curious to know what ingredients are in this miracle product. Nothing even remotely likely to promote weight loss except for caffeine and related xanthines. Drinking lots of coffee is probably just as effective.

A recent issue of the Natural Medicines Comprehensive Database newsletter said:

Akavar 20/50 is a new supplement promoted for weight loss. It contains a long list of ingredients, including large amounts of caffeine from yerba mate, guarana, green tea, and kola nut extracts. It also contains damiana, ginger, schisandra, scutellaria, vitamin B6, magnesium, and other ingredients. Some research suggests that a few of these ingredients might help for weight loss, but this is preliminary. There is no proof that this specific combination of ingredients is effective. Product advertising says, “Eat all you want and still lose weight. . . .” Remind patients that if it sounds too good to be true, it probably is.

And remember, the Natural Medicines Comprehensive Database could say that in print if it weren’t true—but they wouldn’t!

Gary Schwartz believes many things. He believes in psychics, mediums, and life after death, and he believes there is scientific evidence to support these beliefs. Schwartz is now focusing his powers of belief on a new field: energy medicine. In a new book, The Energy Healing Experiments: Science Reveals Our Natural Power to Heal, he explains that we all emit human energy fields, that we can sense each other’s fields, and that healers can influence these fields to heal illnesses and injury. He believes these are not just theories but scientifically supported facts.

The book starts with three “gee-whiz” testimonials of supposed energy healing (which are frankly not very convincing and could be easily outdone by any self-respecting purveyor of quack remedies). He goes on to describe experiments done in his own lab that he claims establish not only our ability to detect and alter human energy fields but our ability to detect the thoughts and intentions of others. In the final part of the book, he descends into blethering about quantum physics, the oneness of the universe, the connectedness of all things, and the possibility that energy awareness will solve all of mankind’s problems.

He claims to have demonstrated many things. First, he claims to have shown that a subject can sense when a researcher’s hand is being held over his or her own hand and can sense when the researcher’s hands are being held near his or her ears from behind. Other experiments supposedly show that people can tell when someone is looking at them or thinking about them. He goes on to describe purported measurements of subtle human energy emissions, Reiki influences on lab cultures of bacteria, and photography of biophoton emission from plants, among other phenomena of dubious reality or significance.

He makes a big deal of the fact that humans emit electromagnetic energy (as picked up by EKG, EEG, etc.), and he would like to think energy healers can pick up that energy and decode it in the same way your radio picks up Rush Limbaugh out of the atmosphere. And then he would like to think that energy healers can send something back into the patient’s body to enable healing. He misses the crucial fact that there is information encoded in the electromagnetic waves your radio detects, but there is no reason to think there is any analogous information coming from the body, much less any way to change that information and send it back to produce healing. I only wish we could use “energy healing” on radio and TV waves to improve the quality of programming!

He makes a big deal of the fact that everything affects everything else. He seems to mean this in a holistic, metaphysical, New Age, “the universe is one and is conscious and we can create our own reality” sense. Science recognizes that small events can have far-reaching effects, but that doesn’t mean one thing can predict or control another. The flap of a butterfly’s wings may set up initial atmospheric conditions that will result in a tornado somewhere else, but that doesn’t mean you can predict the tornado or deliberately use a butterfly to cause one. Theoretically, a change in the magnitude or position of your body mass will enter into the overall gravity equations of the universe, but that doesn’t mean one thing can control or predict another. You could hardly expect to meaningfully influence someone out there beyond Alpha Centauri by losing ten pounds or moving to Antarctica. You can’t expect to change the EEG of an astronaut in the Space Station by exercising to change your own EKG. We are talking about very small influences. If a gnat pushes an elephant, it’s not likely to fall over; it’s not likely to even notice. And then there are inconvenient complications like quantum theory and chaos theory.

The only thing of substance in the book is the experiments, which lose credibility because they were not accepted for publication in mainstream peer-reviewed journals. Schwartz claims this is because of politics. He says prestigious journals tend to reject positive-energy studies. He doesn’t believe that his studies could have been rejected because they didn’t meet the standards of good science. I feel sorry for him: he’s a smart guy, he means well, he really believes he has found something wonderful, but he has a blind spot and just doesn’t get it when others try to point out the flaws in his experimental methods and reasoning. (See Ray Hyman, “How Not to Test Mediums: Critiquing the Afterlife Experiments,” Skeptical Inquirer, January/February 2003, and the follow-up exchange between Schwartz and Hyman, May/June 2003, plus the critical letters to the editor in that issue.)

To put the accusation of “politics” into perspective, consider the Helicobacter experiments. When researchers first suggested that ulcers might be caused by bacteria, they were laughed at. They published their results, peer review had a field day, other labs looked into the idea, more data came in, results from various lines of research coalesced, and within a mere ten years it became standard practice to treat ulcers with antibiotics. It didn’t matter that the idea sounded crazy at first; science responded to good evidence. (See Kimball C. Atwood IV, “Bacteria, Ulcers, and Ostracism,” Skeptical Inquirer, November/December 2004.) If Schwartz had evidence of equal quality, he would get an equal hearing by the scientific community.

Sure, Schwartz has some data that he finds convincing. So did the discoverers of N-rays, polywater, and cold fusion. Good science demands that we withhold judgment until data can be replicated in other labs and validated by other methods—especially when the data come from a researcher as clearly prejudiced as Schwartz. Even the best researchers can fall prey to errors of unconscious bias and unrecognized pitfalls in experimental design.

A good scientist considers the entire body of available evidence, not just the claims of one group of researchers. Schwartz only describes experiments that support his beliefs. Not until the end of the book does he even bring up the fact that other experiments have directly contradicted his findings. He finally gets around to mentioning Emily Rosa’s landmark experiment, published in the Journal of the American Medical Association in 1998, which showed that therapeutic touch practitioners could not sense human energy fields as they claimed. She tested twenty-one experienced practitioners of therapeutic touch.1 They all thought they could detect Rosa’s human energy field and feel whether she was holding her hand over their right or left hand, but when they were prevented from seeing where her hand was, their performance was no better than chance.

Rosa was nine years old at the time, and the article grew out of her school science fair project. The experiment was beautiful in its simplicity. Adult true believers had published much research on the techniques and effects of therapeutic touch, but in the true spirit of childlike questioning, Rosa went back to basics and asked the crucial question: “Is the phenomenon itself real? Can they really feel something or is it possible they are fooling themselves?” Amazingly, no researcher had ever asked that question before. They had ignored one of the basic principles of the scientific method as explained by Karl Popper: it’s easy to find confirmation for any hypothesis, but every genuine test of a hypothesis is an attempt to falsify it.

Schwartz dismisses her experiment as having five “potential problems”:

1. It was a science-fair project done by a young girl.
2. She was the only experimenter.
3. She randomized by flipping a coin, which he calls “an unreliable procedure.”
4. One of the authors was the founder of Quackwatch.
5. The subjects did worse than chance.

These objections are just silly; they are either inaccurate or are ad hominem attacks:

1. It shouldn’t make any difference whether Rosa was a young girl or an old man or a sentient purple octopus from an alien planet. It shouldn’t matter whether she did the experiment for an elementary school project, a doctoral dissertation, a Coca Cola commercial, or a government grant. What matters is the quality of the evidence. In this case, her project was well designed and executed, had clearly significant findings, and was of high enough quality to be approved for publication in a prestigious peer-reviewed medical journal.
2. She was not the only experimenter. Others were involved; the experiment was repeated under expert supervision on Scientific American Frontiers. This should preclude any accusations of deliberate cheating or inadvertent failure to follow the protocol properly. Rosa was the only one to carry out the trials, but what would multiple testers have added to the experiment? The results didn’t depend on any special ability or quality of hers, but on the ability of the subjects who claimed they could sense anyone’s energy fields. For the televised trials, they even got to “feel” the “energy” from each of Rosa’s hands and choose which one they wanted her to use in the trials. About half chose her left hand and half her right. No one objected, “I can’t feel energy from either hand.”
3. Flipping a coin is not an “unreliable procedure”—unless the flipper is deliberately cheating. I hope Schwartz didn’t intend to suggest that. The number of heads and tails was approximately equal, and the distribution appeared random. The editors of JAMA found the method acceptable. There are situations where coin-flipping could legitimately be criticized, for instance in psi experiments where researchers are looking for minuscule differences in large bodies of data and even their computerized random number generators have been criticized for not being “perfectly” random. But in this experiment, the results were clearly significant; it is hard to envision how a different method of randomization could have altered the results. The coin flip was only used to determine which of the subject’s hands she would hold her hand over. The subjects claimed to be able to sense energy fields with either hand, so it shouldn’t have made a bit of difference to their perception. Faulty randomization might have allowed the subjects to perceive a pattern and guess, which would have tended to give false positive results rather than the negative results Rosa got.
4. One of the authors, the founder of Quackwatch, was admittedly skeptical of therapeutic touch. Yes, someone with possible bias was indirectly involved in the experiment. If that is an objection, there is an even greater objection to Schwartz’s own experiments: he and his colleagues are all strongly biased toward belief in energy phenomena and they were directly involved in their experiments.
5. It is simply not true that the subjects did “worse than chance.” Their performance was consistent with chance. If they had done worse than chance (significantly worse) that would have tended to support Schwartz’s claim that some kind of effect was present, even though it would have been the reverse of what he claimed to find.

In my opinion, none of these “problems” invalidates the conclusion that the therapeutic touch practitioners failed to do what they claimed they could do. And if he thinks these were valid problems, why didn’t he simply repeat her experiment in his own lab with multiple experimenters and a more reliable method of randomization? He could have published a failed replication study, and the scientific community could have proceeded to evaluate both studies and sort out the truth. In reality, Rosa’s experiment was a great example of a young child being able to see more clearly than prejudiced adults—a real “Emperor’s New Clothes” story.

I see a lot of “potential problems” in Schwartz’s research—not just ad hominem problems but flaws of experimental design. To start with his most basic experiment: his subjects were blindfolded, sat facing the experimenter with their hands on their laps, and tried to detect which hand the experimenter was holding his hand over. The experimenter held his hands together between trials to keep his hand temperature constant. The subjects often didn’t think they could tell, but they were asked to guess, and their guesses were statistically significant.

The first problem is that blindfolds don’t work. Rosa knew this. Instead, she had her subjects put their arms through holes in a screen and covered the gaps with a towel to preclude any possibility of conscious or unconscious visual cues. She also had subjects lay their arms on a table instead of on their laps, thus reducing the chance of their detecting subtle clues from the person sitting in front of them. Another problem is that when the researcher holds his hands together, that raises the skin temperature and raises the possibility that heat is being detected rather than any other type of energy. And if Schwartz’s results are real, independent researchers should be able to replicate them using the same protocol. Apparently they have not been replicated elsewhere. In fact, Rosa’s experiment amounts to an independent attempt to replicate Schwartz’s basic experiment, only with better controls; and it failed to confirm his results.

If a rigorous scientist thought he had found evidence that people could detect “human energy fields,” he would maintain a healthy skepticism; he would immediately try to prove himself wrong, and he would enlist his colleagues to help show him where he might have gone wrong. He would try to rule out all other possible explanations (the subject might be sensing heat, sound, motion, air currents, might be able to see under the blindfold, etc.). If the phenomenon proved robust, he would try to refine his understanding by doing things like varying the distance to see if it obeyed the inverse square law and interposing a sheet of cardboard or glass to see if the effect could be blocked. Then he would try to use instruments to measure what kind of energy was being sensed.

When a believer thinks he has found something to justify his belief, his approach tends to be less rigorous. Instead of subjecting his original experiment to outside scrutiny, he tends to do more new experiments to try to convince others that he is right. Schwartz goes off on a tangent doing other experiments that purportedly show that the subject is not sensing the energy field but is actually sensing the conscious intention of the experimenter. In one, he claims to show that persons can tell whether someone standing behind them is staring at their head or at their back! If he really believed energy medicine was some kind of psychic thought transmission, he would concentrate on that route of research, but instead he keeps trying to document the ability to detect measurable physical energy fields. His thinking is confused, and he’s trying to eat his cake and have it too.

Schwartz’s style of reasoning was revealed when an experiment to influence E. coli bacteria with Reiki didn’t produce the desired results. Instead of accepting that it didn’t work, he tried to find a way to make the experiment look like it worked. He did some inappropriate “data mining” and tried to show that before the trials where the Reiki practitioners apparently failed, they had been under more stress than before the trials where they apparently succeeded.

He finds a gifted individual who can detect whether a wooden box has a rock in it or not—his success rate is 95 percent for natural crystals, although barely chance for manmade crystals. Unfortunately, before this individual can be tested properly in an independent lab, he develops medical problems and loses his ability. (It’s strange how often these inconvenient things happen when psychic claims are involved.)
Schwartz is mystified by the work of John of God, the Brazilian spiritual healer who performs bloodless, painless surgery. He doesn’t recognize that this charlatan is merely using old gimmicks from the carnival sideshow repertoire to fool the gullible. Schwartz also believes science has established that the human mind can change the pH of water over long distances. He is far less skeptical about such claims than the average scientist.

Schwartz has tried to bolster his credibility by getting a former Surgeon General’s endorsement. In Richard Carmona’s foreword, he says he has seen things he can’t easily explain and says we don’t have all the answers. He helped establish the National Center for Complementary and Alternative Medicine (NCCAM, which he curiously refers to as the National Center for Alternative and Complementary Medicine). The purpose of the NCCAM was allegedly to test complementary and alternative medicine (CAM) and find out which treatments worked and reject those that didn’t. But in its entire history, despite consistently negative results, it has never dared to reject anything. Carmona is currently CEO of Canyon Ranch Health, where Schwartz is the Director of Development of Energy Healing. Canyon Ranch offers integrative medical wellness services, including therapeutic touch. Carmona says, “Where the science supports these integrative concepts of energy medicine, let’s use them. Where there is not enough science, let the studies begin and continue.”

What about “if there is no convincing science or plausible mechanism to support them, let’s stop wasting our time chasing moonbeams”? All of energy medicine hinges on one basic claim: that people can detect subtle human energy fields. If Schwartz is wrong about that, the rest of the claims for so-called “energy medicine” fizzle away.

Since 1996, the James Randi Educational Foundation (JREF) has offered a substantial reward (currently $1,000,000) to anyone who can demonstrate an ability to detect a “human energy field” under conditions similar to those of Rosa’s study. Of the more than 80,000 American therapeutic touch practitioners who claim to have such ability, only one person attempted to demonstrate it. She failed. The JREF challenge is admittedly not a definitive scientific test, but prudence would seem to dictate that if no one can even meet this simple challenge, we shouldn’t be wasting research money on what is probably a myth.

Others have attempted to establish the “science” of energy medicine and have failed.2 Even the NCCAM, which is willing to consider almost any possibility in alternative medicine, is skeptical. It distinguishes between real energy (sound waves, electromagnetism, and other energies measurable by physicists) and the kind of “putative” energy Schwartz is trying to validate. It concludes that the “putative” energy approaches “are among the most controversial of CAM practices because neither the external energy fields nor their therapeutic effects have been demonstrated convincingly by any biophysical means.”

Schwartz sounds like a scientist. He tries to talk the talk and walk the walk. He even makes some skeptical noises to try to convince us he is objective. But there is also a lot of very unscientific language in his book.

For instance:

Human rage and pain, especially generated by terrorism and war, create a global energetic climate whose negative effects can extend from the physical and environmental—potentially including climate—to the psychological and ultimately spiritual. . . . [P]ollution is not simply chemical, it is ultimately energy based and therefore conscious as well.

Really? Conscious pollution? So maybe if we talk nice to pollution it will cooperate and go away? Or should we try doing Reiki to lower the atmospheric CO2 levels? Does Al Gore know about this?

“Energy medicine” is an emperor whose new clothes still look awfully transparent to critical thinkers and to the scientific community no matter what glorious colors and fabrics Schwartz and his colleagues imagine they are seeing.

Notes:

1. “Therapeutic touch” is a bit of a misnomer because these practitioners don’t actually touch but simply massage the air a few inches from the patient’s body. They are convinced that they are detecting and manipulating the energy field, balancing and smoothing it, and correcting any abnormalities, thus allowing the body to heal itself.
2. Hall, H. 2005. A review of Energy Medicine: The Scientific Basis. Skeptic 11(3): 89–93. Available at quackfiles.blogspot.com.
3. nccam.nih.gov.

References:

• Rosa, L., E. Rosa, L. Sarner, and S. Barrett. 1998. A close look at therapeutic touch. Journal of the American Medical Association. 279:1005–1010. Schwartz, Gary E., with William L. Simon. 2007. The Energy Healing Experiments: Science Reveals Our Natural Power to Heal. New York: Atria Books.