Proving Grounds

Research on people helps shape medical care, but not all human studies are alike. Understanding the strengths and weaknesses of the different types of investigations helps researchers, doctors, and the public weigh the evidence for promising treatments.

The news offered hope for those who fear Alzheimer's disease (AD): People who reported taking common nonsteroidal anti-inflammatory drugs (NSAIDs)--a class of drugs that includes ibuprofen--appeared to resist the devastating scourge. But clinical trials subsequently dashed the dream when they revealed that subjects in the early stages of AD lost cognitive function at the same rate whether or not doctors gave them NSAIDs such as naproxen or the prescription drug Vioxx.

But the discrepancy doesn't necessarily mean that NSAIDs are dead as a strategy to ward off AD. The apparently contradictory results came from studies with dramatically different designs. In the clinical trials, for example, patients already had AD. Perhaps they were already past the point at which NSAIDs could help--a point that researchers are taking into account as they design new trials to see whether NSAIDs prevent AD in healthy patients. Such conflicting outcomes can be confusing. "It's very hard for the public to sort out," says oncologist Norman (Ned) Sharpless of the University of North Carolina, Chapel Hill. "It's even hard for physicians." But by weighing the pros and cons of different experimental schemes, researchers can devise the best protocol for testing the benefits of any new therapy. And an understanding of what the various human studies can and can't tell us can also help patients make sense of the breakthroughs that they hear about on the news.

The first clues about how factors such as drugs, diet, or exercise affect health often come from observation. In observational studies, clinicians note the health and behavior of a group of people but don't otherwise intervene. These studies can look backward or forward in time. In a retrospective study, researchers ask participants or comb through historical records for information about the past. For example, investigators might query patients with cardiovascular disease about their previous diet and exercise habits--and then compare their answers to those of people with healthy hearts. In prospective studies, investigators track a group of people into the future, cataloging their behavior over months or years, and noting which individuals eventually develop, say, heart disease or dementia. Behaviors that are more common in one group than in the other might indicate risk factors or identify potential treatments: Perhaps people with heart problems tend to gobble more than their share of doughnuts, or those whose brains remain sharp tend to have a wide social network.

These research tactics tip off investigators to factors that might cause or prevent disease, but they have limitations. Participants don't always produce reliable accounts of their behavior. In addition, these observations don't prove that a certain activity causes a particular disease; for instance, a study could reveal that people who down multiple martinis each day are likely to get lung cancer--but only because people who drink are also more likely to smoke.

Moreover, habits vary widely in most groups, making it difficult to pinpoint a single factor responsible for a disease. For instance, scientists have found that people who take vitamin E resist heart disease. But such individuals might eat better, exercise more regularly, and visit their doctor more often than do people who don't consume supplements. So researchers must go beyond observational studies to prove that a treatment or a lifestyle choice alters the course of a disease. That's when they move to clinical trials.

Clinical trials--in which researchers conduct controlled experimental tests in people--are the gold standard for assessing the safety and effectiveness of drugs or other interventions. In these studies, investigators dictate exactly how much and what treatments study subjects receive--factors that, in observational studies, are left to chance. Clinical trials trump observational studies because they eliminate factors that might confound or bias the results.

First, subjects are placed randomly into groups that either do or do not receive the test intervention. This randomization ensures that other factors--such as age, weight, socioeconomic status, and gender--are represented equally in the two groups. The strongest clinical trials are also "placebo-controlled." In these trials, volunteers who do not receive the test drug get an inactive substitute instead. Thanks to this design, researchers can attribute the benefits of a potential drug to the compound itself rather than the "placebo effect"--the psychological boost of receiving a treatment. Factoring out the placebo effect can be important: In some studies, 30% of patients in the placebo group show improvement.

For a placebo-controlled trial to be effective, patients can't know which group they are in. And in the very best trials, the doctor doesn't know either; such studies are called double-blind. That lack of information further removes any bias that might skew results. For instance, if a doctor knew that a subject were getting the drug, he might inadvertently drop hints to that effect. "Even in the way doctors greet patients, there's a bias that's unconsciously transmitted," says neurologist Robert Friedland of Case Western Reserve University in Cleveland, Ohio. And patients who think they're getting the drug might show improvement due to the placebo effect. In addition, doctors who know which patients are receiving treatment might downplay side effects in that group because they're optimistic about the drug's benefits, says Sharpless. Both situations mean that a drug would appear more effective than it is.

Clinical trials often follow up on promising results from observational studies. But they sometimes fail to confirm the initial findings, perhaps because the clues offered by observational studies aren't explicit directions--and they sometimes lead researchers to a dead end. For instance, the observational studies on AD and anti-inflammatory drugs lumped together different NSAIDs. Perhaps the clinical trials, which focused on a few specific compounds, showed no effect because researchers picked the wrong drugs, says AD researcher Leon Thal of the University of California, San Diego. Further clinical studies might uncover particular NSAIDs that do prevent or treat AD.But conducting more studies means investing even more time and money--and devoting more time to recruiting volunteers, which can be a formidable task. As researchers continue to investigate various approaches to battle AD, volunteers "are becoming a scarce resource," says bioethicist and AD researcher Jason Karlawish of the University of Pennsylvania in Philadelphia. And in any disease that affects the elderly, researchers struggle to find patients who can fulfill the demands of a clinical trial and still represent potential users of the drug (see "Seniors Can Find Standard Meds Hard to Swallow" and "Test Patterns").

Despite these challenges, researchers need to pursue human studies to refine disease treatments or prevention. Meanwhile, premature excitement over promising but inconclusive study outcomes will likely remain a problem. "People don't want to wait for randomized trials or very well done prospectively gathered data," says Sharpless. "It's very easy to believe the hype that a compound is going to be great. It's very hard to impress upon people that sometimes you just have to wait. It's not appropriate to treat people with drugs until [we really show that] they're beneficial." But a finer-tuned sense of what makes data strong will help everyone recognize truly good news when it arrives.

R. John Davenport is a science writer based in Santa Cruz, California, and an associate editor of SAGE Crossroads' sister site, SAGE KE. He'd like to participate in a clinical trial on the health benefits of ice cream--but not if he's in the control group.