Challenges to Using an Evidence-based Approach to Complementary and Alternative Medicine

The task of approaching complementary and alternative medicine (CAM) modalities using an evidence-based approach presents several challenges, the most common of which are listed below.

Access to the literature

When performing searches in MEDLINE and other databases, literature evaluating CAM modalities may not be retrieved due to the poor indexing of these topics by the database producers. Many studies are published in journals that are not indexed in MEDLINE and may remain undiscovered unless practioners actively seek them out. Even when discovered the journals may not be readily available to researchers. As part of the EBCAM initiative we are helping to catalog the extensive literature in non-MEDLINE journals that is available through the Kelly Library at the New England School of Acupuncture (NESA), our project collaborator.

Types of studies reported in the searchable literature

Many of the articles written about CAM modalities have been case series and reports of clinical experience. This data can help decide if a modality is safe, if enough experience in the literature is reported. However, this data will not answer whether the modality is effective, a conclusion that results from the use of the randomized controlled trial (RCT). With the increasing use of CAM modalities, there are more and more RCTs and subsequent systematic reviews appearing in the searchable literature in databases such as MEDLINE and the Cochrane Database of Systematic Reviews (CDSR).

Lack of standardization of practices and variablity between studies

Even where RCTs are used to study a particular CAM modality, systematically reviewing a body of literature can be difficult because of the variability in practice from one trial to the next. Timing of interventions, variability in the dosing of treatment preparations, specific type of modality (traditional acupuncture vs. electro-acupuncture) are all important considerations when looking at different studies even with similar interventions and outcome variables. Because the specific issues regarding dosing, timing have not been tested against each other there is no "standard of practice" which governs these modalities. This variability also affects the usefulness of systematic reviews as studies that investigate the same modality for the same outcome may NOT be fully comparable.

Different medical systems have different classifications for disease states

For instance, the diagnosis of hypertension [HTN] is not classified as a separate entity in East Asian Medicine and is considered a symptom rather than a separate disease. Therefore, patients with hypertension who are cared for by East Asian Medicine providers will have different treatments depending on their non-allopathic diagnosis. This can make the generalizability of a CAM modality to a large population within the western model difficult .

Control group issues

To perform a high quality RCT, there needs to be a blinded control group that one compares to the treatment group. In performing some of the treatments involved in CAM, it is difficult to completely blind a control group to a “placebo” intervention. This is particularly the case when there is a physical intervention such as Tai Chi rather than for a nutritional supplement, which can follow the same experimental designs as those used for pharmaceutical trials.

Resources and Links

What is Complementary and Alternative Medicine (CAM)?
How the National Center for Complementary and Alternative Medicine (NCCAM) defines CAM..
Evaluating complementary medicine: methodological challenges of randomised controlled trials.
Su Mason, Philip Tovey, and Andrew F Long. BMJ 2002; 325: 832-834. [PDF]
Cochrane Collaboration CM (Complementary Medicine) Field
Based at University of Maryland at Baltimore, the Cochrane Collaboration CM offers a newsletter, a searchable registry of randomized controlled trials (RCT's) in CAM.
BMJ.Com Complementary Medicine
Collection of articles published in BMJ on CAM topics. Coverage from 1998-present.
EBM Online Complementary Medicine
Collection of CAM related articles published in BMJ and 20 plus other journals made available by BMJ EBM Online service.
Bandolier—Complementary and Alternative Medicine
Provides abstracts of systematic reviews and meta analyses of complementary and alternative therapies.
Series of articles published in the Journal of Alternative and Complementary Medicine on teaching evidence-based complementary and alternative medicine. Current journal subscription available at the Hirsh Health Sciences Library not available online.
1. Mills EJ. Hollyer T. Guyatt G. Ross CP. Saranchuk R. Wilson K. Evidence-Based Complementary and Alternative Medicine Working Group. Teaching evidence-based complementary and alternative medicine: 1. A learning structure for clinical decision changes. Journal of Alternative & Complementary Medicine. 8(2):207-14, 2002 Apr.
2. Wilson K. Mills EJ. Hollyer T. Vohra S. Guyatt G. Evidence-based Complementary and Alternative Medicine Working Group. Teaching evidence-based complementary and alternative medicine: 2. A conceptual approach to causation-part 1. Journal of Alternative & Complementary Medicine. 8(3):379-83, 2002 Jun.
3. Wilson K. Mills EJ. Hollyer T. Vohra S. Guyatt G. Teaching evidence-based complementary and alternative medicine: 2. A conceptual approach to causation-part 2. Journal of Alternative & Complementary Medicine. 8(3):385-9, 2002 Jun.
4. Wilson K. McGowan J. Guyatt G. Mills EJ. Evidence-based Complementary and Alternative Medicine Working Group. Teaching evidence-based complementary and alternative medicine: 3. Asking the questions and identifying the information. Journal of Alternative & Complementary Medicine. 8(4):499-506, 2002 Aug.
5. Wilson K. Mills EJ. Ross C. Guyatt G. Teaching evidence-based complementary and alternative medicine: 4. Appraising the evidence for papers on therapy. Journal of Alternative & Complementary Medicine. 8(5):673-9, 2002 Oct.