Archive for the ‘Clinical Research’ Category
There has always been, and will likely always be, a disconnect between clinical practice and the results of well-designed research trials in the world of alternative medicine. This has become an increasingly obvious fact over the past several years as alternative medicine has become incorporated into even our top-tier academic medical centers. Proponents of alternative medicine even have their very own government-funded research agency, the National Center for Complementary and Alternative Medicine, which puts over 100 millions dollars to use studying things like energy healing and homeopathy every year. And on a seemingly weekly basis, press releases and credulous news reports emerge with claims of new scientific research proving that believers in these unconventional modalities have been right all along. The curmudgeonly skeptics are often portrayed as foolish, or worse, closed-minded.
What the vast majority of the patients seeking help from these snake-oil salesman don’t seem to realize, or at least don’t seem to care, is that alternative medicine is a house of cards built upon a foundation consisting of anecdotes, testimonials, cherry picked research practically designed to yield positive results, and the interference of lawmakers. Many physicians, now commonly referred to as “shruggies”, are uninterested in the reality behind alternative medicine and a growing minority actively support it. Naturally there are also a number of well-known celebrity practitioners and supporters of these therapies that serve as spokespersons for the industry. But what is unknown to the shruggies, and ignored or downplayed by the silver-tongued alternative medicine gurus, with their persuasive abilities eerily similar to leaders of religious movements, is that in many cases the definitive studies have been done, the question of efficacy answered. They may claim to integrate the best of both conventional and alternative medicine, but they know where their bread is buttered. So they forge ahead, touting anecdotes and inadequate research as proof. Proof that they don’t even need because their followers don’t really care what science has to say on the subject. Reality takes a backseat to belief.
The most common incursion of alternative medicine into published journals, most of which have impact factors only appreciable with the aid of a high-powered magnification device, is in the form of the case study. A rigorously documented case study can be a valuable thing, even if its place in the hierarchy of evidence is above only the undocumented anecdote and testimonial. They can be especially useful in determining that an all-inclusive assumption is in error, the classic example being that the existence of even one black swan disproves the statement that all swans are white. For an example in the practice of medicine, a single case report documenting Sickle Cell disease in a caucasian child of European ancestry is enough to prove that Sickle Cell disease does not occur only in patients with African ancestry. Case reports can also be helpful, especially in a series, in guiding future research. And in instances of rare conditions, case reports may be all that is available to guide therapy. But a case report, regardless of how well it is documented or of how good a journal it is published in, is incapable of answering the question of cause and effect. They can lead a researcher to a come up with a dandy hypothesis or, as occurs with much higher frequency, to waste time and money on a wild goose chase. A physician simply can’t say that a treatment works based on case reports and, as with personal anecdotes, the whole of the medical literature on a subject must be taken into account.
I was diagnosed with Tourette Syndrome when I was about seven years old. Bear with me, I promise that this isn’t a non sequitur. I have been very lucky in that my Tourette’s, after a rocky few years in middle school and early high school, is fairly mild. It is obvious to most people that I have a movement disorder, but it has never impacted my ability to function in society and to succeed in my chosen profession of pediatric medicine. In fact, I often think of my Tourette’s as a positive aspect of my life, believing that it helped shape who I am as a person. I feel that it helped instill in me positive personality attributes that are beneficial in the practice of medicine, such as empathy and compassion.
I have never needed treatment of any kind for my Tourette’s, but I have tried to keep up with the literature on various therapies, pharmaceutical and otherwise, for the condition. I usually stick to reputable sources but every now and then I like to go “slumming” in the world of chiropractic research. During my last trip across the border between science and pseudoscience, I discovered a gem which perfectly illustrates the trouble with case reports and with chiropractic. I stumbled on a press release for a case report published in late 2009 in the Journal of Pediatric, Maternal & Family Health – Chiropractic involving the year-long treatment of Tourette’s in a 20-year-old female. Not unexpectedly, the results were reported as positive. A quick google search using “chiropractic for Tourette’s” revealed almost 100,000 results, with many (I stopped counting at 100) individual chiropractors referencing this case report when calling for patients with Tourette’s to come in for treatment. They were not equivocal on the subject either but then again practitioners of alternative medicine don’t much care for nuance and subtlety. Warning bells should go off whenever a proposed abnormality, be it the chiropractic subluxation, stagnant chi, or liver flukes, causes every problem and has a ready made solution. So chiropractors are claiming to be able to cure or significantly impact the symptoms of Tourette’s, and this is the proof. Or is it?
The answer is of course no but I’ll explain. Tourette’s is a condition perfectly suited for placebo effects. It waxes and wanes in severity, with many sufferers experiencing periods of time during which their tics aren’t as forceful or frequent. Tourette’s is generally at its peak in early adolescence, or about ten years after diagnosis for most patients, but, and I hope this clearly shows why using such a case report as anything other than for lining your child’s hamster cage is granting it too much importance, two-thirds of Tourette’s patients experience remission or significant amelioration of their tics in late adolescence or early adulthood. The patient described in this case report was 20-years-old and the treatment took place over an entire year. It is very possible, if not highly likely, that the patient’s symptoms would have improved regardless of the therapy she sought out. Not taking into account the natural course of a condition tends to occur when you lack experience in diagnosing and caring for patients with that condition, and when your background in pediatric pathology involves a weekend course held at the airport Howard Johnson. I’d say its a rookie mistake, but chiropractic was invented in 1895.
The tics, whether motor or vocal, simple or complex, experienced by Tourette’s patients are unique. They can be suppressed, for a time, only to come back with a vengeance eventually. They are suggestible. They are effected by stress and by fatigue. And, like pain, there is a considerable subjective component to the movements and outbursts. One patient’s suffering with the condition may be much greater than another’s despite having the same tics in the same location. The potential factors at play are too numerous to list but to give one example, my most obvious tic, where I forcefully jerk my left arm outward and vigorously flex and extend my arm several times until it “feels right” doesn’t bother me when I’m sitting in bed reading a good book, my bedside lamp safely out of reach. But ask me how I feel when sitting in a crowded movie theater with a stranger on my left who I really don’t want to elbow in the face. Now what if I had a job that required long shifts with close quarters, perhaps on an assembly line of some sort. Life could be pretty miserable. Life might suddenly become much less miserable if I was moved to a different location that allowed more breathing room. I might report a definite improvement in my quality of life, decreased stress, and subsequent decrease in severity of tics simply because my boss took my condition into account.
We don’t know what else was going on in the life of the patient in this case report. We don’t know about changes in her life that might have decreased her anxiety about the condition. Perhaps simply receiving a hands on treatment from a perceived authority figure certain that they can provide help caused a decrease in anxiety that led to her improvement. We don’t know if her improvement was simply because of the natural course of her Tourette’s. All we know is that she reported an improvement after a year of chiropractic care. Correlation does not necessarily equal causation as the old saying goes.
Another aspect of placebo involves reporting bias. Patients who are emotionally invested in a therapy are more likely to report positively. Patients who like their physician or other practitioner, such as a chiropractor for instance, are more likely to report positively. Patients in a study, or who are going to be written about in a case report, are more likely to report positively in order to please the person in charge who stands to gain from positive results. There are other well established psychological entities, such as cognitive dissonance, that motivate positive reporting. Nobody wants to feel foolish, for instance, wasting a year of their life and the money required to do so on a failed therapy. None of these can be accounted for in case reports such as this.
But is this case report even useful as a hypothesis generator? Does it support the authors’ calls for more research into the chiropractic subluxation as the etiology of Tourette’s and chiropractic spinal manipulation as a treatment? Once again, the answer is no. While the exact cause of Tourette’s is not established, there is expert consensus that it has a genetic origin and is a disorder involving the connections between the cortex and subcortex leading to abnormal synaptic neurotransmission. Some experts have raised the possibility of an autoimmune etiology, particularly related to infection with Group A Strep, but this is controversial. There is absolutely no legitimate concern regarding the possibility that impairment of spinal nerves, as is claimed to occur in the presence of chiropractic subluxation, is a factor. Sure there have been instances where experts have been wrong, and as a skeptic I always leave room for surprising discoveries, but the evidence required to overturn the current consensus would need to be extraordinary. For the chiropractic community to raise this possibility based on an isolated case report this worthless is laughable.
Here we go again.
Seemingly unbeknownst to acupuncture researchers around the world, the question of efficacy beyond that of placebo of inserting small needles millimeters into a patient’s skin has been answered by a number of large, well-designed, randomized and appropriately blinded clinical studies. Acupuncture has simply not been shown to be effective for any condition. The trail leading up to these trials is littered with anecdotal evidence, poorly conceived methodologies and weak treatment effects, not to mention a complete lack of a plausible mechanism of action for a near totality of the claims of benefit. But a disheartening number of people, including medical professionals at prestigious academic centers, have been duped.
This is, unfortunately, what we have come to expect with an inert therapy surrounded by a number of placebo inducing trappings such as undeserved positive press coverage, meritless state licensure, true-believing practitioners confidently spouting mystical or pseudoscientific jargon, and the laying on of hands in a relaxing environment. Just look at the practice of chiropractic, which is equally unscientific and unproven outside of acute lower back pain, for which it has been shown to be as good as but not better than conventional therapy (or doing nothing!). You can’t go 5 blocks in any direction without seeing a sign for one of these offices, usually containing a silly slogan like “Chiropractic Keeps My Spine Off My Nerves!” or “Got Chiropractic?”. But back to acupuncture.
The reality of acupuncture research is a frustrating one for medically oriented skeptics such as myself. We have difficulty seeing the world through the eyes of a true-believer in a treatment modality, because belief based on anecdotes and ideology, rather than acceptance based on plausibility and evidence, is so foreign to the way we think. If the evidence doesn’t support the practice, why does it continue? And why do researchers, like Dr. Florian Pfab, of the Technical University of Munich, continue to churn out studies like the one recently reported on in Reuters Health involving the use of acupuncture for atopic dermatitis? The answer to the first question is simple: reason is insufficient to change a thought pattern not brought about by reason in the first place. In regards to the second, I fear that these researchers know on some level that the only means of producing positive scientific support for their already firmly held belief is to ignore the foundations of science-based medicine.
The study being credulously reported on by Amy Norton of Reuters Health, most likely inspired by a press release sent by the parties involved, seems to have been published twice. It appeared in a supplement to the November 2008 European Journal of Integrative Medicine and was again accepted for publication in the European Journal of Allergy and Clinical Immunology (Allergy) in late 2009. In both abstracts, the background is essentially identical. One calls itch “a complex and unpleasant sensory experience that induces the urge to scratch”, and the other refers to it as simply “a major symptom of allergic skin disease.” Both abstracts make the unfounded claim that acupuncture has been shown to have an “effect” (the earlier version called this effect positive) on histamine-induced itch in healthy volunteers. What has been shown is that itch, like pain, nausea and pretty much any symptom with a predominantly subjective component, responds to placebo and/or distraction.
Both abstracts describe the investigation of acupuncture on allergen-induced itch and wheal/flare response in a double-blind, randomized, placebo-controlled, crossover trial. It is quite unclear from the abstracts, which is all I have access to unfortunately, whether this actually occurred. Both studies involved exposing 30 subjects with eczema to an allergen stimulus, in this case a skin prick containing house dust mite or grass pollen, both before and after receiving one of two experimental treatments, or nothing. The two experimental treatments involved so-called true or verum acupuncture, where needles were placed in specific points linked by traditional chinese medicine to improving this sort of problem, and sham acupuncture, where needles were placed in points not associated with benefit in cases of allergic skin reactions. The “no acupuncture” group was provided complementary issues of the European Journal of Integrative Medicine or something equally unlikely to improve a subjects health. Okay, I made that part up.
The severity of the itch in all three groups was recorded on visual analog scales in both studies. The difference between the publications becomes apparent in the next step. In the earlier version, after 10 minutes the size of any wheal/flare reaction was measured in addition to skin temperature, and an itch questionnaire was completed by the patient. In the more recent publication, wheal/flare size was also recorded but now skin perfusion as measured by LASER-Doppler, and not skin temperature, was included. The questionnaire results were also included.
Both abstracts report that mean itch intensity was decreased in the true acupuncture group compared to the sham and no acupuncture groups, and that sham acupuncture was better than none. The results, reported as significantly lower still fall close enough for there to be a distinct possibility that, based on the simple fact that were only around ten participants in each arm of the trial, there was in reality no difference at all between the groups. But taken at face value, the more rational interpretation is that there are varying degrees of placebo effects. We don’t know based on the abstract whether or not the person performing the acupuncture was blinded or if the blinding of subjects was sufficient. Based on the track record of this type of research, and on the design of another study by the lead author, I have my doubts. All it would take to lead to this kind of result would be for those receiving sham acupuncture to know it based upon practitioner cues. This issue could be resolved by asking participants which arm they thought they were in upon completion of the trial.
Both true and sham acupuncture were found to “significantly” decrease subsequent itch when given prior to allergen exposure compared to not receiving acupuncture. This isn’t suprising as placebo is already well established as effective in ameliorating the subjective experience of itching. It is also well established that placebo can effect the appearance of an allergic response. So it is also not suprising that wheal and flare size was smaller in the true and sham acupuncture group compared to doing nothing. But, once again, with such small study numbers, these results are essentially meaningless. Much larger and better designed studies, with innovative methods of obtaining reliable acupuncture placebo, have already been performed which look at a variety of outcomes. They show that not only does it not matter where you put the needles, it doesn’t matter if you even insert the needles. I am still left wondering what is the point of all this.
The earlier version of the study found no difference in skin temperature between groups after exposure to allergen. Suspiciously in the second version of what is the same study otherwise, skin perfusion rather than temperature was measured. The authors report that skin perfusion, which is being used as a marker for allergic response, was significantly decreased in the true acupuncture group compared to no acupuncture. Skin perfusion for the sham acupuncture isn’t included in the abstract. This difference between the two, otherwise identical, studies is problematic. Why would the authors not include the negative skin temp results in the second version and insert the claimed positive skin perfusion results. Is this simply a means of squeezing multiple publications out of one study, which is commonly done and quite shady in my opinion, or is it that the authors omitted negative results because of pro-acupuncture bias. What other outcome measures were obtained? Were they negative as well?
The conclusion of the authors is naturally positive, and also completely irrelevant. Conclusions are irrelevant in all studies. Many times they don’t jibe with the results. For instance, one recent acupuncture study showed that true acupuncture was no better than placebo. The authors claimed that both worked! They followed with the standard study boilerplate call for further investigation into why placebo acupuncture is effective. As I mentioned at the onset of this post, the questions being asked by acupuncture research have been answered. Most of them, as in the study on eczema, didn’t warrant being asked. Small studies with questionable or absent blinding are almost always wrong, especially if the treatment effect is small and the mechanism of action is implausible.
The Reuters Health article, also based on the early view online release of the paper in Allergy which I based this post on, and email correspondence with lead author Florian Pfab, is typical of mainstream coverage of alternative medicine. It gives a two sentence explanation of what eczema is, glosses over methodology, quotes the author, refers to acupuncture as ancient but throws out a possible but unproven “modern explanation”, and ends with the ubiquitous call for more research. A simple phone call to one of many well-known skeptical resources on acupuncture, or a quick google search for that matter, was all that would have been necessary to avoid such a worthless piece of pseudojournalism.