How to Evaluate a Clinical Study: Guidelines to Help Understand Published Research

Science is a work in progress. Every day, promising studies are released and new treatments are approved. But how can you evaluate whether the information you're reading is relevant to your own health or to that of a patient, friend, or family member?The newly emerging field of evidence-based medicine provides a set of straightforward guidelines for evaluating the data presented in published clinical studies. But since these studies don't come with a set of instructions on how to understand them, the following is a set of criteria you can use to help evaluate what it is you're reading.Why is it important to learn how to read medical studies? Isn't that knowledge that your doctor should have?We believe educating patients on how to be their own best advocate is important because no one medical provider can know everything about a particular disease. By better understanding medical evidence, patients can (1) be better critical evaluators, (2) help others understand the science, and (3) be a more active team member when working with doctors to make treatment decisions.Not all studies are well designed, or even well written. Study authors will sometimes make claims or recommendations not supported by the data. And often there is a filter of conflicting interests, often from drug companies or the media, which stands between you and the study itself. This filter can misinterpret or misrepresent the facts.For many of us, when reading a medical study, we often read what we want to see. Our eyes are drawn first to the conclusions presented in a study's abstract, a summary usually printed at the front. Often, we want to know how our health can benefit from this study right now. But before accepting the abstract's conclusions at face value, we suggest the following:First, go to the very end of the study, just before the references section, and look for a very short paragraph usually titled "Disclosures" or "Conflicts of Interest" to find out who funded the study. Pharmaceutical industry-funded trials often tend to be biased or over-interpretive of the true results. A meta-analysis published this year in the Journal of the American Medical Association (JAMA) found that industry-sponsored trials were 3.6 times more likely to show positive results, in favor of their product. (Bekelman, 2003)Second, examine the study authors. Do they have the credentials or experience to properly conduct the study? The most important credential is actual clinical experience in the question being investigated.Third, go beyond just the abstract to read the details of how the study was conducted. An average of 30 to 40% of published studies misrepresent the actual findings in the abstract, according to a recent critical analysis of six major medical journals. (Pitkin, 1999) Look carefully at the section usually titled" Materials & Methods." While reading this section, ask yourself the following questions:» Are the patients in the study closely matched to you in terms of age, gender, and specific medical problem?» Were patients randomly assigned to the different treatment groups being compared?» Was the trial double-blinded (neither patients nor doctors knew whether patients were receiving the treatment or placebo)?Fourth, read the section usually titled "Results." Are the study's conclusions, as described in the abstract, supported by the actual data?Lastly, for literature promoting a particular product or treatment, is the report you are reading original work or just "borrowed science," taken from other studies not actually conducted by the authors?Whether reading to learn about your own medical condition or on behalf of somebody else, you can be an effective consumer and critic of medical information and contribute to thediscussion and decision-making process regarding the health issues that concern you.REFERENCES:· Bekelman, J. E., Y. Li, et al. (2003). "Scope and impact of financial conflicts of interest in biomedical research: a systematic review." JAMA 289(4): 454-65.· Greenhalgh, T. (2001). How to Read a Paper: The Basics of Evidence-Based Medicine. London, BMJ Publishing Group.· Pitkin, R. M., M. A. Branagan, et al. (1999). "Accuracy of data in abstracts of published research articles." JAMA 281(12): 1110-1.Top of Page....................................................................................................................

Article Summary: Assessing Study Quality - Red Flags to Watch For

1. Who funded the study? Could there be a conflict of interest?2. Do the investigators have proper credentials to do the work?3. Who are the subjects being studied?4. Is the trial randomized and double blinded?5. Do the actual study results support the author's conclusions?6. Is it original work or "borrowed science"?Top of Page....................................................................................................................

What is Evidence-Based Medicine?

In the practice of evidence-based medicine, the reader applies critical appraisal skills to reading clinical studies and asks:1. Precisely what type of patient is the study about?2. Exactly which treatment is being tested?3. What are the specific outcomes being measured? For example, for our meta-analysis on lung cancer, rather than ask, "do herbs work for lung cancer?"" we asked, "do patients with non-small cell lung cancer, using Chinese herbal medicine in combination with Taxol/Carboplatin chemotherapy, have better short-term results, lower toxicity, and better long-term survival than those patients using the same chemotherapy alone?"Evidence-based medicine also encourages critical thinking on the part of medical investigators; it compels study authors to design studies carefully with a precisely defined clinical question, analyze their findings properly, report results honestly, and make recommendations that are wholly supported by the data.Top of Page....................................................................................................................

How Can I Develop More Critical Reading Skills?

The British Medical Journal has published a concise, easily comprehensible guide called, "How to Read a Paper: The Basics of Evidence-Based Medicine." Available in paperback from www.bmjbookshop.com, it introduces you to simple, yet practical criteria with which to take a critical eye to medical literature.