The media had a feeding frenzy when a data-mining report from Yale researchers published in JAMA Oncology suggested a causal relationship between use of complementary medicine and shortened life span among cancer patients. A New York Times subheading was representative: “People who used herbs, acupuncture and other complementary treatments tended to die earlier than those who didn’t.”
The appearance of this apparent death knell due to the “nostrums” as the Times writer called them – “herbs, vitamins, traditional Chinese medicine, homeopathy, naturopathy, yoga, acupuncture and others” – had a fascinating timeliness. The publication of “Complementary Medicine, Refusal of Cancer Therapy, and Survival among Cancer Patients with Curable Cancers” arrived within months of news that the uber-mainstream American Society for Clinical Oncology endorsed integrative breast cancer guidelines that feature many of the same “complementary” approaches.
Given the estimates of life lost among the complementary medicine users reported by the study’s lead authors Skyler Johnson, MD, James Yu, MD, MHS and colleagues, ASCO would seem to be aiding and abetting a rather nefarious process.
What gives? I reached out to leading integrative oncologists and then, on the suggestion of one, to statisticians – and then to the lead author. What I found raises questions as to whether JAMA Oncology should retract the paper – and if the Times and other media outlet follow with their own mea culpas. An undeniable conclusion is that the poorly vetted research has provoked a damaging sort of fake news if one assumes such a study should help the public and clinicians know whether to recommend and use complementary and integrative practices.
Two Integrative oncologist perspectives
University of California at San Francisco professor of clinical medicine and integrative oncologist Donald Abrams, MD was familiar with the study when I reached him. He knew that the author team had published a 2017 report mining the same USA National Cancer Database that concluded lifespan shortened in those who say no to standard care and instead turn to “alternative” methods.
Abrams replied to an electronic query via a long paragraph attached to an email. These were the main points.
- The definition of “complementary” (practices used to complement usual oncologic care such as chemo and radiation) was confounded with “alternative” (therapies used instead of).
- He noted that even the authors note that if one removes those who have delayed treatment or
refused treatment – thus who would seem to view their CM as “alternative” rather than purely “complementary” – that “all of the differences go away.” - He questioned the statistical basis: “Really, out of 1,901,557 people in the cohort [of cancer patients from the USA National Cancer Database], they only find 258 using these oddly defined treatments?” He knew that multiple surveys have established that at least 30%-50% of people with cancer use complementary medicine.
- Abrams called “retro and uninformed” the assumption that complementary services are provided by “non-medical providers.” Many licensed practitioners are providing the services, including many medical doctors and nurses in scores of top academic medical centers.
In short, fundamental definitions in this article were wrong and the statistical work overlaid on them were out of whack with all prior research. Abrams concluded with perhaps more gentleness than this combined perspective would seem to deserve: “A poor paper in my opinion.”
Lynda Balneaves, RN, PhD, the current president of the Society for Integrative Oncology (SIO) – the organization that produced the original guidelines that led to the ASCO-endorsed breast cancer guidelines noted above – began her response generously. She shared that the SIO “always encourages people to conduct research in the area of complementary medicine, It’s Important to take advantage of databases, to get more population-based perception.” She added: “We only wish that we had more databases with better designed assessments and measurement of complementary therapies and outcomes for these therapies.”
Then, in the course of a 45-minute interview, Balneaves entered more deeply into the multiple troughs Abrams’ critique had begun to carve in the Yale researchers’ article. Her questions followed the same pattern. She focused on the major “mis-classification bias” of the authors. She adds: “Why would they use a definition of ‘complementary’ from 1998?” She noted that SIO has in fact published a formal “Comprehensive Definition of Integrative Oncology” that was readily available. She also wondered “why didn’t they bring onto their research team someone with expertise in the field?”
These perspectives from the SIO president were echoed in a document provided by Heather Wright, ND,
FABNO, president of the Oncology Association of Naturopathic Physicians (OncANP) which was responsible for its contents. Three naturopathic physicians specializing in oncology were on the team that produced the SIO breast cancer guidelines, including the lead author, Heather Greenlee, ND, PhD.
The OncANP team called out the authors for “trying but failing” to distinguish between “alternative” and “complementary.” They conclude: “The present study found that patients who choose to pursue alternative therapies (not in coordination with oncology care) demonstrate consistently poorer outcomes. The finding is not representative of those patients who choose integrative and complementary cancer care.”
Analysis from a statistician
Balneaves suggested I get the view of a statistician. She led me to contact Scott Mist, PhD, MAcOM, MS, MA, LAc. Mist is an assistant professor for anesthesiology and perioperative medicine at Oregon Health Sciences University School of Medicine. He also serves as the associate editor for statistics at JACM – Paradigm, Practice and Policy Advancing Integrative Health (The Journal of Alternative and Complementary Medicine). I contribute to JACM as editor-in-chief.
Mist e-mailed a bulleted response within hours of the request. Given the importance of clarifying whether complementary therapies are in fact-life life-threatening, I asked Mist to edit the e-mail for broader consumption, which he did. I include it in full below. I focus here on the two point to which Mist most called attention.
First, Mist raises a bright red flag over the decision of Johnson and his team to focus on just 258 of
1,901,815 patient that the authors put in the CM group. Mist points out that, elsewhere in the paper, the authors note that multiple surveys have found that between 48% and 88% of people with various types of cancer use some complementary methods. He does the math: “There should have been between 836,799 and 1,673,597 CM users. But they found [just 258] less than 0.01%. That should have stopped the study right there.” He adds for emphasis: “There is no way that less than 1/100th of the population is representative of the population of CM users.”
Mist’s analysis raises additional questions regarding other extrapolations that appear to be based on questionable assumptions. He suggests a frequently used method for establishing a set of comparators was inappropriately applied. He concludes with what he considered the other most significant issue: “They say that when controlling for refusal rates and delays, there is no difference in the survival rates. Therefore, it can’t be the CM treatments that are the difference.” Game over.
No Response Following Request to the Authors
I sought an interview with Johnson on Friday August 10, 2018, by emailing him Mist’s memo. I shared my plans for a special set of articles to be published August 15, 2018. Then a column in JACM. I re-sent the email to Johnson over the weekend. On Sunday, August 12, I began including his colleague, Yu. As of this moment the weekend is not over. No blame. I have not yet had a reply – understandable on an August weekend. I will update this segment when I hear from them.
Should JAMA Oncology be asked to retract the paper?
Bad papers get published. But rarely does a single paper as poor as this one have the impact that this has had, in terms of media play and wrong-headed public education. Should JAMA Oncology be asked to retract the paper – especially given the damaging message to the public?
I put the question to Balneaves, who noted that usually it is either ethical issues or major fatal methodological flaws. She wasn’t ready to make such a call. Mist shared his reflections by email: “Honestly I don’t know. I personally wouldn’t have accepted it simply because there are two missing analyses and the database is incomplete given that around half [of people] are using CM but only 0.01% were registered as using CM.” He continued: “If it were me, I would have made them do the analyses, make alternative hypotheses of why it might be so, and say that this is a preliminary study that needs a prospective study to see if the relationship holds true. I would place serious money that it wouldn’t hold true.”
Then in an email that immediately followed Mist added: “And I don’t like wagering money.”
Chad Aschtgen, ND, FABNO co-founded Seattle Integrative Oncology through which he and his colleagues have agreements with Swedish Medical Center and other leading Seattle area medical establishments to help manage their patient interested in integrative care. After ticking off the familiar litany of challenges to the JAMA Oncology paper, Aschtgen he drove home the negative effect of its publication: “It gives the wrong message to practitioners who are beginning to see the value to their patients in complementary therapies, and to the patients whose lives can benefit from them.”
Coda: I have bias here. I have had cancer – tonsillar cancer – and for it, took 33 radiation treatments and three cisplatin blasts, a treatment plan that University of Washington endowed professor and head and neck surgery chair Neal Futran, MD, DMD, once transparently described to me as “one of the most brutal things we do.” His colleagues successfully burned the cancer away – and concurrently caused and left me with multiple adverse effects. So I also am among the 48%-88% who use complementary therapies and practitioners. (In fact, Dr. Aschtgen is my naturopathic oncology expert with whom I presently stil consult from time to time.) It would take many moons to share all the ways I experience as beneficial the multitude of practices and practitioners from which I have benefitted over the 9 years since my diagnosis, relative to the damage and sequellae of that brutality.
I asked many who I interviewed if they believed that bias informed the development, the review process, and decision to publish this article. I can just say that the consensus was that it is a shame that good professionals, with separate expertise, were not talking to each other when this study was developed. That can be remedied, Dr. Johnson, and Dr. Yu. But the damage done to conventional physician and patient perspectives by the publication will not be undone unless JAMA Oncology sets itself to making amends for the adverse effects of its publication decision.
Learn more HERE
