For many of us a “sham” is not a good thing. It conjures images of fraud and being duped.
Last month was a good example as the publication of three separate studies — all using sham surgeries in their design — brought into question the benefits of two expensive and widely used treatments, as well as one very controversial treatment. Here are those studies:
If a patient gets better or worse from a surgery, how do you know if you can attribute either result to the targeted intervention (like placing a stent or removing some ‘abnormality’)? If they get worse couldn’t it be partly attributed to other risks of surgery … like anesthesia, infection, or post-operative complications? And if they get better couldn’t that be a placebo effect?
Evaluating a surgical intervention isn’t as straightforward as a drug study where you can compare the active medication to a sugar pill in order to tease out the true impact.
But with a sham surgery it is more straightforward. That’s because in a typical sham surgery study you randomize one group to get the full surgery (like in the ORBITA trial above, to have a stent placed). The sham group, on the other hand, goes through all the same preparation and surgical steps except for one … they don’t get the intervention being studied (ie. no stent placed). Neither the patients emerging from surgery, nor the doctors evaluating the outcomes of the patients, know who did — or did not — get the stents. The confounding contributions of bias, placebo effect, and surgical complications are nullified.
You now have a control: the sham group. And you’ve created a double-blind, randomized controlled trial (or, RCT, for short) — the gold standard of medical evidence.
Why are these sham surgery studies so important? Let’s take the coronary stent and shoulder decompression studies first.
Before these studies were published last month, the interventions evaluated in both studies had been performed on millions of people, costing billions of dollars, and been accepted as routine procedures in certain clinical situations. If these new results are supported by future RCT’s (and yes, they are needed in all 3 cases above) then these costly (or lucrative, depending on your perspective) treatments may become discredited and abandoned.
These are called ‘medical reversals.’
In the case of coronary stents we’re talking about potentially debunking an established treatment that goes back nearly 40 years. That raises important questions. Not the least of which is: who should be held accountable when it takes decades to show that a widely implemented treatment may have been not just costing us quite a bit, but accomplishing very little?
“I think the burden of proof should be on those who created these interventions, popularized them, and stand to make money from them,” says Adam Cifu, MD, an internist at the University of Chicago who specializes in evidence-based medicine, and who — along with Vinay Prasad, MD — coined the term ‘medical reversal’ in the book Ending Medical Reversal: Improving Outcomes, Saving Lives .
“But instead the burden of proof falls to other researchers, many of them quite brave, who are left to prove that maybe these innovations don’t really work. Our very first obligation is to do no harm, but until these drawn out reversals occur the harms are huge. People could have been benefiting from other therapies. Sometimes these therapies — and stents are a good example — can cause direct harms. And finally, there’s the immeasurable cost of losing the faith, respect, and trust of our patients. I’ve experienced that and it’s very painful.”
There are critics who argue that sham surgery is unethical. They say a fake surgery is not in the best interest of the those in the control group; that is, that surgery should only be performed to benefit people. Furthermore, the surgery puts them at risk, and this violates one of the most hallowed tenets of medicine: first do no harm.
I asked Cifu what he thinks of critics who call sham surgeries unethical.
“Yes, there are risks to sham surgeries like any other surgeries,” says Cifu. “But think of the difference between putting a few people at minimal risk versus putting hundreds of thousands of people at more significant risk for many years to come.” He adds:
“We’re strangely uncomfortable with upfront costs in our culture. We’re talking about a little extra care, a little extra time, and a little extra money on the front end to make sure these treatments and interventions are safe. Why won’t we accept these costs, but we seem to accept decades of harms from not being careful in the first place?”
An area of medicine that is particularly prone to both breakthrough hype and subsequent medical reversal is orthopedic surgery. As in the case of shoulder decompression listed above, other orthopedic procedures have come under fire because of sham surgery studies. These include injecting specialized cement to stabilize spine fractures caused by osteoporosis, as well as arthroscopic knee surgery to fix wear-and-tear changes of the meniscus (the C-shaped cartilage that forms an important cushion inside the knee joint).
I ask Dr. Julie Switzer, an orthopedic surgeon at the University of Minnesota, why that is.
“So often we are operating on the assumption that people’s pain can be explained by the abnormal findings we find on an x-ray or MRI,” says Switzer. “But we can’t always know that the things like arthritic changes, or unstable joints, or ligament tears that we can change with surgery are actually responsible for that pain.”
She also thinks there can be a powerful placebo effect with surgery when you can show a patient an abnormality on a scan (unlike depression, let’s say, where no such abnormality can be pinpointed), and couple that with an objective surgical procedure (unlike an antidepressant pill), that can be very convincing for patients. She adds:
“There’s a unique investment that patients feel with surgery that introduces optimism, expectation, and hope. A sense that the benefit will be much greater than simply taking a pill. And that the greater the risk you feel you are taking, the greater is your investment in the outcome.”
One of the reversals listed above — the so-called ‘liberation treatment’ for MS — is instructive on many levels.
Dr. Paolo Zamboni — a vascular surgeon motivated by his own wife’s struggle with MS — developed a stenting procedure to improve venous drainage of the brain, which he felt was impaired in the majority of patients with the neurodegenerative disorder. He dubbed this mechanism “chronic cerebrospinal venous insufficiency (CCSVI)” and the procedure as “liberation treatment.”
His preliminary data suggested improvement in as many as 3 out of 4 subjects after the procedure. This launched not just a frenzy of news coverage, but also a social media tidal wave driven by MS patients and MS patient advocacy foundations. Clinics popped up across the globe to offer the procedure. Canadians even demanded the national health care system cover the costs.
But gradually, as the burden of proof fell to other researchers, it became clear the procedure really didn’t work. But, as eloquently reported by Helen Branswell of STAT, the damage had been done:
“Millions of dollars were spent trying to see if Zamboni was correct. Untold research hours were diverted from other lines of MS inquiry. Previously collaborative relationships between doctors and patients, as well as patients and foundations set up to advocate for their needs, were damaged.”
Let me be clear here: I’m not blaming Zamboni for this. In fact, he should be applauded for following up his preliminary, unblinded 2009 study with the sham RCT study published two weeks ago, and concluding: “this procedure cannot be recommended for the treatment of patients with MS.”
But all the fallout does raise another question …
As Branswell points out, some news outlets were riding the breakthrough bandwagon … along with a lot of vulnerable patients, a slew of advocacy groups, a wave of medical tourism, and a fair number of doctors.
Was this misguided ripple effect avoidable? Yes. Early critics of Zamboni’s 2009 study clearly pointed out the limitations: an unblinded study of just 65 patients with subjective endpoints and lack of sham control. Zamboni himself cautioned: “The results of this pilot study warrant a subsequent randomized control study.”
But in a time where believing can take the place of knowing, and breakthroughs make better copy than caveats, Zamboni’s ‘liberation treatment’ should serve as a cautionary tale for all of us who cover new therapies; especially for diseases light on treatments, but heavy in suffering. Quite a bit is at stake.
“It would be wonderful if the media wasn’t so smitten with new breakthroughs,” says internist Adam Cifu. “Because that makes them partially responsible for boosting these therapies before they’re adequately tested.”
“Some sort of ‘breakthrough’ care that is going to make us better is what all of us find exciting. It makes patients excited and hopeful. Doctors get excited that they can help. And the media gets excited about the clicks and ratings that come with the hype. That’s exacerbated by the pressure of adopting a new idea quickly without adequate testing. Unfortunately, it’s not terribly exciting to say ‘let’s slow down, be careful, and let’s make sure that before we release a treatment that it actually works.’ But that’s exactly what we need on the front end. “
As journalists this ‘front end’ is our responsibility. Whether it be in how we frame a news release or how cautious we are in approaching breaking news. Awareness of this — as well as awareness that medical reversals are a necessary and inevitable part of medical progress — will go a long way toward helping us do what our parents recommended to many of us: Take your time and do it right the first time. Because if you don’t, it’s a lot harder to fix further down the road.
And, yes, people can get hurt.