If You Know Shuzi Like the Merseyside Skeptics Know Shuzi: Testing the Shuzi Sports Band Video
September 26, 2012
To the uninitiated, they look like ordinary black bracelets. And that, it would seem, is exactly what some so-called sports bands are – despite their hefty price tag. – The Daily Mail, 3rd September 2012.
The Merseyside Skeptics recently launched a video called Testing the Shuzi Sports Band. In the video, you can see Vice President Michael Marshall and other members of the Merseyside Skeptics put the Shuzi Qi band, a so-called athletic enhancement bracelet to the test, from the Merseyside Skeptics website:
Sports performance technology manufacturers Shuzi Qi came under fire today after product tests revealed their performance-enhancing wristband to be ineffective... Despite marketing claims that the product aids a player’s performance, the demonstration showed that when a player is unsure which band he’s wearing, the £60 product makes no discernible difference.
In order to find out more about its creation, I spoke to Mike Hall of the Merseyside Skeptics; we’d met previously at the QEDCon in 2010. The convention will be returning to Manchester in 2012 from April 13 - 14.
Kylie Sturgess: Firstly, for those persons who don't know what a sports performance wristband is, what do they involve?
Mike Hall: These bands have been on the market for a few years now, and the manufacturers claim that they interact with the body's bio fields by some mechanism. In the case of the Power Balance Bracelet for example, which is probably the most famous one, they have a special hologram that interacts with the bio field of the body—whatever that is. And somehow (by mechanisms they usually don't adequately explain) this improves an athlete's concentration and performance. Balance was the big one, hence the name “Power Balance” and that sort of thing.
And then they marketed these things to athletes. And usually the way they market these things to athletes is to get notable athletes to wear the things and thus aspiring athletes will see them and go, “Oh, well, famous athlete X wears a wristband and therefore I must also wear this band.”
They've been in the market for a few years now, and as I say, Power Balance was the most famous one, but skeptics in Australia have done fantastic work in kind of chasing Power Balance out of the country. I think that the story goes that the TGA demanded that they publish something on the website saying that they still don't work, after the TGA investigated, that these bands have the evidence lacking.
Power Balance ended up going bankrupt and quitting the country. And they have published something on the website saying they still don't work, but this has kind of left a powerful vacuum in the sports performance band area that all these copycat bands have rushed in with their own technology and tried to fill. They say, “Yeah, yeah, yeah, yeah. Power Balance, that was all nonsense, but our thing does the same thing, but ours actually works.”
Sturgess: There isn't that much of a difference between them then? In the video where you deal with the Merseyside Skeptics, you investigated the Shuzi band. Aren’t they all kind of the same?
Hall: They're all broadly the same as best as I can see. In the case of Shuzi, they say that the special chip in the band interacts with your body's bio field in order to give you all these performance benefits in your sporting ability. In the case of Power Balance, it was a hologram. And there are various others, power bands, and various others on the market and they all claim they have some pseudotechnological explanation for what their band does and why it works.
In the case of Shuzi, they say that their microchip uses “nanovibrational technology,” which is pretty much three scientists standing some words in a line!
The claim is that they “unclump your blood” and they use known pseudoscientific techniques to demonstrate that this is the case. They use live blood analysis, which is a long debunked nonsense diagnostic that actually doesn't work at all as far as I'm aware. Then, they apply kinesiology—which is, I believe, actually an old vaudeville act that was appropriated in the 1960s by a chiropractor—in order to promote this sort of stuff!
Then there's the balance test that people are likely to be familiar with, the one where you stand on one leg, they push your arm, and you fall forward. But then they put the magic band on you and they do it again and suddenly you don't fall over.
Basically, it's a magic trick, but it's one of those magic tricks that you can perform without necessarily knowing that you're doing it. Kind of like cold reading in that you can fool yourself into thinking that you're not actually doing a magic trick, but in reality, you're being manipulated through use of the ideomotor effect.
But yes, as best I can see they're all more or less the same. They all have a different piece of pseudo-technological nonsense to explain them: “We have a special piece of technology that interacts with your body in some way to make it better,” in whatever way it is they lay claim.
Sturgess: The Merseyside Skeptics have recently produced a video testing the Shuzi bands—was it difficult finding participants for it?
Hall: No, we're very lucky in terms of finding participants for it. The story goes is we have a tip off from the Australians, again, who said, “Shuzi is coming to the UK," and I believe Shuzi already has withdrawn from Australia as a result of criticism? Not only that, I learned that their head distribution channel is going to be based in Liverpool. Which is in Mersey, right? So, it's on my patch and if you're going to set up on my patch, well, you're going to have to deal with me!
Michael Marshall (a.k.a. Marsh), the Vice President of the Merseyside Skeptics, initially tried to engage with Shuzi; he called them up and said, “You know, we're very interested in your bands, but we're not sure—you know, we'd like to put it to the test and see if it works.”
And Shuzi seemed very receptive to that idea and said, “Yes, we'd be interested in working with you on a test,” and then suddenly stopped returning calls!
So Marsh would call them up and these calls would just ring out and go to voicemail and he would send them emails and they wouldn't get back to him. And then, one of the guys at Merseyside Skeptics, a guy called Warren, said, “Well, if they're not willing to cooperate on a test, why don't we just do the test. We could do the test.”
So Marsh called me up and said, “Why don't we just devise a test?” Now, as luck would have it, my brother is a rugby player. And as luck would also have it, my brother is the team’s kicker!
And in a further stroke of luck, my dad is the groundsman at the local rugby club! So I was able to get access to the grounds. I was able to get access to a rugby player, and I was also able to get access to a rugby field.
I liked the idea of using kicks, because it was a binding outcome. It was either there was a goal or there wasn't a goal. It wasn't something like how far can he kick it, since there's a subjective thing in that, or how good was the kick and all that. It was a good, solid, yes or no, black or white; a binary outcome.
I was on a phone call to Marsh, I said, “Oh, I've got this fantastic idea. Why don't we get my brother in? My dad can let us use the pitch. My brother can do the kicks. And kicks are good, they're a binary outcome. We'll do this test.”
Then I phoned my brother up and said, “Look, are you interested? Do you think this is the right thing to do?” And he was a bit hesitant at committing to a hundred kicks because normally, when he's practicing, he'd do maybe twenty. So a hundred was a big deal for him! But yes, he quite readily agreed once I'd explained to him what we were trying to do.
Sturgess: Alice Howarth, the physiologist from the University of Liverpool, said in the video that it was not a statistically significant difference in terms of performance when the band was worn and when the band wasn't worn. What was the test that was used?
Hall: We used a chisquared analysis that was suggested to us by Chris French, Professor Chris French at Goldsmith's, University of London, who we're very friendly with at Merseyside Skeptics. He's a good friend. And so, when we were devising the test, we got on to Chris and we said, “Look, is a hundred kicks enough for this to be statistically significant results?” And, “What statistical techniques should we be using to analyze this data?”
And Chris suggested that we use the chisquared analysis, which when we ran the figures through the chisquared analysis we came out with a P-value of 0.5.
My understanding a P-value is limited but it has to be a value of 0.05, so actually an entire order of magnitude difference before you could consider the results to be statistically significant.
Sturgess: Do you think that the results will generalize to athletes developing skepticism about other products? I mean, colored tape, protein supplements, shoes and other kinds of performance enhancers, that Marsh mentioned at the start of the video?
Hall: It certainly generalized that for my brother, who in the aftermath of the test said to me, “That was very, very exhausting and I was hating it towards the end. But ultimately, I'm really interested in this kind of thing,” because as a semiprofessional sportsman, he encounters this sort of nonsense all the time. Whether it is like the energy drinks or the whey protein shakes, or just even things like people having lucky socks.
For example, if you go into the changing rooms at these rugby clubs, everybody gets dressed in a particular order. They’ll come in wearing their street clothes, and then they'll strip off completely from the waist down and leave the top half of their clothes on and then put their left sock on and their left boot on, and then the right sock and the right boot on and put their shorts on over the top of that. Then, once they've gone that far, that's when they start taking off their top in order to put the shirt on. And it's just a superstitious thing! “Oh, the last time we played and I did really well, I put my sock and my boot on and then my sock and then my boot and I didn't take my shirt off until later.”
And everybody does this and that in a different order. And so, my brother, who is a bit skeptically inclined himself, just from listening to me rant and rave about it over family dinners, observes these behaviors in his club and says, “This is obviously, clearly nonsense.” However, these superstitions go on in sports all the time. And he's now really fascinated. He wants to know how much of this is true and how much of this is nonsense. Can we come up with tests for these behaviors? Can we come up with tests for prescription shoes? And can we come up with tests for getting dressed in an appropriate order and all that sort of stuff?
So he's very enthusiastic about it. I don't know if we can come up with tests possible for other items, but it certainly seems to have engaged his interest. Whether he'll engage in that more formally, I have no idea!
Sturgess: Do you have any other tests planned? I mean, I know the Merseyside Skeptics are incredibly proactive out there in Liverpool.
Hall: We've got nothing planned at the moment in terms of in this area of sports. What we are planning to do because the video was always meant to be a piece of public engagement—it was really never meant to be a science experiment. It was meant to be something that we could put out there and as a piece of public engagement with science. And so, what we said up front is we wanted it to be about as statistically representative as the average Mythbusters episode. That's the angle we were kind of pitching it at. But it's got to be rigorous enough to be convincing but not so rigorous as to make it useless as a piece of public engagement.
Having actually finished the test, and looked back at the protocol that we came up with, we are actually thinking of writing it up and seeing if we can get it published in a journal. With the numbers involved, you've only got one player who does a hundred kicks, the numbers involved aren't that spectacular, but the methodology is so solid.
We had a doubleblind protocol and the way that the protocol worked is we bought two of these Shuzi bands and I took one of the bands and drilled the microchip out of it. So what Shuzi claimed was the active ingredient was therefore not present in one of these bands. So I removed the microchip from one of them and then covered the hole where the microchip was with electrical tape on both bands so you couldn't visually tell them apart.
Those two bands were then handed over to Alice, unaware of which one was the control band and which one was the active band. Alice then split the hundred kicks into ten blocks of ten and used a random number generator to decide which band was for which test so something like: band X, and then Y, Y, Y, Y, X, X, X, Y, X, X, Y (where we'd labeled them X and Y).
Alice was then responsible for attaching the band, with no idea whether she was attaching an active band or control; she only knew X or Y.
David, my brother, was our player who would go out onto the pitch and, unaware of what band he was wearing, even if he knew whether it was X or Y, he wouldn't know whether X was the real band or the control band. Then the people gathering data on the pitch didn't have any idea whether it was X or Y, or if X or Y was the appropriate thing.
We were blind right across the board, and I wasn't permitted to talk to Alice for the duration of the test, and Alice wasn't allowed out of the changing room to interact with people on the pitch during the test.
We were quite rigorous about it because, at the end of the day, we took it really seriously. We could have fudged the results completely in our favor if we wanted to, but that would have been intellectually dishonest, and it's not something that we were interested in doing.
If there was a genuine effect that would have actually been cool, but there wasn't, and we didn't really expect there to be. But we still wanted to make sure that our biases didn't influence the test like that, so we were quite rigorous with the methodology of the test.
There was never really a question of putting that level of detail into the video because we think that, actually, as a piece of public engagement, would probably turn people off. For the science community, the people who are interested in that, we are going to publish the full protocol, and it's going to be up on the Merseyside Skeptics website.
Sturgess: I look forward to hearing the public's response to it; it’s appearing in newspapers?
Hall: We've had commitments from a few newspapers in the UK. They're going to run it on their websites and embed the video, which hopefully can show how much oomph we can make with this in terms of the public and see Shuzi's response to it as well.
We went to Shuzi last week and said, “Look, we went ahead and did this test anyway and it turned out that your band didn't work any better than a control band. What have you got to say about that?” So far, they haven't actually come back with any sort of response.
So it will be interesting to see how Shuzi react to that, because I imagine they're going to have journalists of all stripes on the phone to them for the rest of the week saying, “Well, what do you have to say about this?”
Fingers crossed. Hopefully we'll be able to make a splash with this.
Sturgess: Most unskeptically, fingers crossed!
Hall: Ah, the irony of saying fingers crossed as a skeptic!
Sturgess: I know—it's brilliant! Thank you so much for talking to me, Mike Hall.
The Merseyside Skeptics site can be found at http://www.merseysideskeptics.org.uk.
In 2013, QEDCon will be held in Manchester from April 13 - 14, with more details at http://www.qedcon.org.