In Focus: Learning Health Care Systems

Every day, physicians and other clinicians around the country record millions of observations and treatment decisions in electronic medical records—creating a vast repository of information that is increasingly being used by clinical researchers to answer practical questions about the safety, effectiveness, and value of health care services.

As these researchers turn data from routine clinical care into knowledge and knowledge into guidance for physicians at the point of care, they help create what the Institute of Medicine has called "learning health care systems."¹  Such systems can enrich the findings of randomized control trials by allowing researchers and clinicians to learn about the effectiveness of treatments for patients who can't realistically be studied in trials—including those with multiple comorbid conditions, pregnant women, and patients undergoing surgery. Because many of these learning systems rely on previously collected data, they can also generate discoveries quickly and efficiently.

To facilitate the development of such systems, a number of government agencies including the Centers for Disease Control and Prevention (CDC), the U.S. Food and Drug Administration (FDA), and the Centers for Medicare and Medicaid Services (CMS) have been funding researchers who mine the data amassed by health systems and insurers to monitor the safety of drugs, devices, and vaccines after they've been approved and put to use. Data from electronic health records and administrative claims are also used by leading health care organizations, specialty societies, and companies to predict which patients are at most risk of developing certain conditions or experiencing adverse events, to conduct comparative effectiveness research, and to create guidelines and decision support tools that can be iteratively improved with feedback from practicing clinicians. The latter are particularly helpful in intensive care units and others locations where critical details about a patient's condition may be overlooked in cases of provider overload.

One of the first and most robust learning health care systems was created by the HMO Research Network, which was formally established in 1994 by a consortium of health care systems that now includes Kaiser Permanente, Group Health Cooperative, and Geisinger Health System, among others.²  The health care systems maintain clinical information on some 15 million patients, which enables their research divisions to collaborate with one another and with external researchers to answer questions about the prevalence of different diseases, the comparative effectiveness of treatments, and the safety of drugs and vaccines. 

Because of its size and the availability of rich clinical information, the system enables researchers to learn about subgroups of patients that can be otherwise hard to study, such as the 50 percent of Native Americans who receive care outside of the Indian Health Service, individuals with diabetes in their 90s, and pregnant women undergoing chemotherapy.³  And since its member organizations are located in different states, the network also allows researchers to study the impact of local health policy and payment design on health outcomes. 

Two of the main sponsors and beneficiaries of the network's experience collating and analyzing data are the CDC, which relies on the network to monitor vaccine safety, and the FDA, which through its Mini-Sentinel Program has been able to monitor adverse events linked to specific drugs. The latter program, led by Richard Platt, M.D., of Harvard Pilgrim Health Care Institute and Harvard Medical School, can quickly investigate potential risks that may crop up as drugs are prescribed to many more patients (with more underlying conditions) than are typically involved in premarket trials. 

The Mini-Sentinel Program recently assessed whether a new rotavirus vaccine put children at risk of developing a condition in which part of a child's intestines slides into itself, potentially leading to serious complications or death if untreated. Back in 2000, the Rotashield vaccine was withdrawn from the market after it had been found to cause this condition, and the FDA thereafter required the premarketing trial for any new rotavirus vaccine to involve at least 60,000 children. With the Mini-Sentinel database, researchers were able to expand the pool of patients studied to 1,000,000 kids. The analysis found that the new rotavirus vaccine led to a slightly increased risk of developing this intestinal complication (about one in 100,000). The FDA decided this was an acceptable level of risk when weighed against the potential for developing rotavirus, though the agency did require the product information to be changed to help parents and clinicians make informed decisions. 

Identifying At-Risk Patients 

Such large data sets are also being used to identify patients who are most likely to suffer side effects or adverse events. The Stanford Center for Biomedical Informatics Research, for example, relied on de-identified data from Stanford University's clinical data warehouse to determine which children with juvenile idiopathic arthritis were most at risk of developing uveitis, an inflammation of the middle chamber of the eye that can lead to blindness. The center looked for evidence to support a hunch from a physician that the problem cropped up in children with allergies and found such a link. "Once we convince doctors that yes, this is true, we want to put this into the EMR so that the question of whether the child has been seen by an ophthalmologist as recommended is asked," says Nigam Shah, M.B.B.S., who heads up the research. 

Decision Support Tools 

Providing information to physicians at the point of care about a patient's individual risk is one of the most valuable benefits of learning health care systems. The health care modeling and data analytics company Archimedes, Inc., a Kaiser Permanente innovation, uses a simulation model, refined over 20 years, to predict not only a patient's risk for having heart attacks, strokes, and diabetes (among other conditions), but also suggests the likely outcomes of different health care treatments and lifestyle choices. This product, IndiGO, which is used by physicians, accountable care organizations, health maintenance organizations, and others, draws on information from clinical trials, health records, literature review, and epidemiology and make predictions about outcomes that change as patients age, develop new conditions, see doctors, and receive treatments that result in various outcomes. 

Some providers use IndiGO to show patients a graphical representation of their risk of developing various conditions and engage them in a discussion of the potential risks and benefits of different treatments, such as statins, which some patients want to stop taking for fear it will cause memory problems later. "I will open up IndiGO and say, 'Here's what happens if you do that. Your risk of a heart attack or stroke in the next five years is going to go from 5 percent to 10 percent. Is that the risk you are willing to take to stop the statin?' When they look at that, they say, 'I will keep taking it,'" says Greg Reicks, M.D., a family practice physician in Grand Junction, Colo., who has incorporated IndiGO into his practice. 

Reducing Provider Overload

By carefully analyzing data in electronic medical records, researchers have also been able to highlight vital clues about a patient's condition. This can be particularly valuable in critical care units where physicians are sifting through hundreds of data points in high-stress situations. To help them, the Mayo Clinic has spent five years developing and is now piloting an electronic medical record viewer known as AWARE (Ambient Warning and Response Evaluation) that seeks to combat information overload by gathering and analyzing data collected from medical records and devices. This information is adjusted for patients' unique characteristics (such as age, weight, and comorbidities) and is used to highlight in real time what is most salient. The system, for instance, alerts clinicians when patients have abnormalities or are at heightened risk of developing critical conditions. 

Its programmers continually add new rules in response to new information—once adjusting the algorithm to alert providers that ICU patients may be at risk of renal failure based on data on urine output collected over two hours instead of six, in response to new findings from clinical investigations. Early trials of the system found that, while the tool synthesized information and reduced time spent reviewing patients' records, it did not disrupt providers' ability to accurately diagnose. 

Point-of-Care Trials 

The technology infrastructure that allows researchers to learn from already-collected data also enables them to establish "point-of-care" clinical trials, which can be used to determine the relative effectiveness of drugs and treatments that have already been proven to be safe. The Veterans Health Administration (VA) is now testing this model to determine which of two methods of administering insulin to diabetics (a sliding-scale regimen versus a weight-based regimen) is more effective. In three facilities, physicians ordering insulin during an office visit using VistA, the VA's electronic health record system, are notified at the point of care that their patient is eligible to participate in the study. If the patient agrees (and more than 90 percent of those asked have done so), the system selects the method and collects outcomes data until there is sufficient evidence to suggest one method is superior to the other. 

A key purpose of the trial is to determine "if clinicians would be comfortable explaining that they don't know which [approach] is best and whether the patient would be comfortable with the computer choosing what medicine they got," says Leonard D'Avolio, Ph.D., associate center director of biomedical informatics for the VA's Massachusetts Veterans Epidemiology Research and Information Center.⁴  In the future, the VA plans to gather research questions and projects from practicing clinicians so that when findings are approved for systemwide dissemination, there will be more confidence in them, D'Avolio says. 

Building Living Clinical Guidelines 

Like those at the VA, physician leaders at Boston Children's Hospital have sought to capture and learn from information about medical practice by creating "living clinical guidelines" that suggest sound practices, based on information from medical literature, but continue to evolve as they accommodate practical experience and new knowledge.⁵  Physicians are free to divert from the guidelines, known as SCAMPs (Standardized Clinical Assessment and Management Plans), and practice patterns and outcomes are studied to inform their next iteration. "Not only do we allow and encourage physicians to divert from the SCAMPs, but we actually capture the reasoning for diverting," says Michael Farias, M.D., a senior pediatrics resident at Boston Children's. "It's a valuable source of information and innovative ideas for how we can improve the SCAMP."

For example, the first version of the SCAMP for treating patients with a dilated aorta suggested they should be referred to a genetic specialist—a recommendation most providers were ignoring. Later analysis indicated that the vast majority of the referral visits did not yield important information, so the next generation of the SCAMP suggested only patients with other signs of an underlying genetic disorder, such as family history or joint dislocations, should be referred.

The iterative guidelines have led to increased adoption. Adherence to SCAMPs at Boston Children's, where the guidelines were rolled out in 2009, is about 80 percent, compared with 39 percent to 53 percent adherence for traditional guidelines.⁶ 

Finding Insights in "Dirty" Data

Not all of this work depends on large investments in health information technology. The Harvard Predictive Medicine Group has demonstrated that rudimentary data, including those drawn from administrative claims on diagnoses, prescription use, and lab tests, are sufficient to flag future clinical risk. Back in 2009, the group's researchers identified patients—on average up to two years in advance—who were at risk of experiencing domestic violence. Ben Reis, Ph.D., director of the program at Harvard Medical School and Boston Children's Hospital, believes using commonly available data is the key to enabling widespread adoption of these sort of research findings. "If it's a choice between 'clean' data and 'dirty' data, we go for the 'dirty' data. If it's a choice between working with the most wired hospital in the world or a hospital that is average, we go with the average hospital," he says.

Challenges Ahead

All of these initiatives—large-scale research networks, risk assessment and clinical support tools, point-of-care trials, and living guidelines—require a "paradigm shift" in medicine, one that redefines the relationship between research and practice as a continuous feedback loop, with evidence and learning flowing in both directions. 

Whether these kinds of open-ended investigations stem from a clinician using electronic databases to support a hunch or a programmer using automated rules to sniff out patterns and trends in "big data," this work will require new analytic methods and vetting processes.

But as medicine continues to grow more complex and providers are increasingly measured according to the value they provide, tools that enable them to learn from routine practice are likely to become more important. "If you look at other industries, you see there's an evolution over time from expert intuition to simple testing and guidelines to mathematical models that enable complex learning systems," says Don Morris, Ph.D., Archimedes' vice president of scientific product and technology development. "Medicine is behind the curve—medicine is cautious, and there are good reasons for that. We are still in a regime where we are trying to do A vs. B comparisons, but when you have 20 trials comparing A vs. B and they don't agree with each other, that causes confusion. Mathematical models are not trying to get yes or no answers. When medicine makes this shift [to learning systems] we will see big increases in quality and efficiency."