A Collaborative Multi-Stakeholder Effort
Sponsored by The Leapfrog Group and Bridges To Excellence
The work contained in this White Paper reflects the efforts from health plans, employers, consultants, and providers to define a set of recommendations—Best Practices—that have the potential to improve the measurement of provider efficiency and the science behind it. While the paper focuses on efficiency, all the contributors acknowledge that measuring efficiency should be done in conjunction with measuring effectiveness of care, so that consumers, purchasers and payers can better understand and identify the value of the services being delivered, and providers can better understand the steps they need to take to improve the value of services offered.
Executive Summary
A decade of escalating health care costs combined with a growing focus by the Institute of Medicine (IOM) and others on the deficiencies in the safety and quality of patient care have created considerable momentum around the concept of measuring both provider clinical quality and provider cost efficiency (cost efficiency from the payer's perspective). The science of measuring physician and hospital quality has advanced considerably in recent years. Organizations like the Joint Commission for the Accreditation of Healthcare Organizations (JCAHO) and the National Committee for Quality Assurance (NCQA) have developed standard measures that are now widely used throughout health care. Many of those measures have been reviewed through the National Quality Forum's consensus-based process and adopted by health plans and rating agencies. As a result, there is good understanding within the industry on how to measure health care quality at various levels (in particular health plan and hospital levels), even if, regrettably, there is not complete uniformity in the application of those measures, or universal achievement of high performance on these measures.The same cannot be said of efforts to measure efficiency. Until now, there lacked a systematic, empirically informed and consensus-based process to understand how best to measure cost efficiency. Instead, each individual stakeholder has had a tendency to approach this effort separately, which has decreased the industry's ability to learn from natural experiments, understand and catalog best practices, and collaborate on relevant research. As a result, organizations that have introduced efficiency measurement initiatives have often been met with resistance from doctors and hospitals on (a) the meaningfulness and validity of the results, and (b) the lack of transparency in the underlying measurement methodologies. However, the need for valid, reliable, and actionable information on provider efficiency remains very high. Payers and purchasers/employers (and increasingly consumers as a result of changed health benefits designs) have understandably had a keen interest in identifying doctors and hospitals that consistently deliver good clinical outcomes without wasting resources, and using that information to support benefit designs, network management and public report cards that, together or separately, might induce patients to choose more efficient providers.
The goal of this White Paper is to launch an ongoing process that will provide guidance to all stakeholders based on available knowledge about efficiency measurement. The guidance is provided in the form of principles and recommendations that are believed to be acceptable to—if not necessarily embraced wholesale by—multiple stakeholders. These recommendations are not intended to represent the "last word" on provider efficiency, as both the art and science of efficiency measurement are still in their infancy and we expect them to grow. Rather, they are intended to create a framework that is sound enough to use as a basis for measurement today, and to act as a catalyst for stimulating the evolution of measurement as our knowledge and understanding of this field grows. To that end, an online learning community has been established at www.regence.com/research to facilitate the continued sharing of knowledge.
Furthermore, during the next few years, the NCQA, a key collaborator in this effort, will develop evaluation methods that will help determine the extent to which health care organizations measure physician and hospital performance following principles set forth in this paper. NCQA is currently working with many national experts and stakeholder representatives in order to publish a first set of evaluation methods (standards) by July 2005. It is expected that NCQA will be able to evaluate health care organizations wanting to demonstrate their adherence to these standards by the latter half of 2005.
NCQA's efforts in this area, referred to as the Quality Plus Initiative, are part of its overall work to refine its evaluation methods to focus on critical areas where health care organizations (managed care organizations and preferred provider organizations) can be expected to significantly add value for their members. For a more detailed discussion of NCQA's methods and implementation timetable see www.ncqa.org/Programs/Qualityplus. Until the NCQA's work is complete, the White Paper will be periodically updated to reflect new knowledge and understanding from real world applications in this field through the continued work of many organizations.
Our underlying belief is that for hospital or physician efficiency measurements to be widely accepted in the market, they should be feasible to implement for health plans, credible and reliable for consumers, and fair, equitable and actionable for providers. That requires certain conditions to be met.
First, it is important to incorporate enough recent data to develop a statistically reliable determination of provider efficiency. If some data elements are unavailable, they should be omitted uniformly to ensure the comparability of diverse data sources. However, some empirical evidence suggests that pharmacy data is important for measuring physician efficiency. Reports should only be issued for physicians or hospitals with substantial reportable cases; we offer suggestions regarding reporting thresholds and evidence supporting the recommendation.
Second, we recommend analyzing the data using industry standard episode grouping methodologies, and applying robust case mix and severity of illness adjustments. Even with standard episode groupings and risk adjustment, it is still important to restrict comparison groups to truly comparable facilities or physicians. To that end, we are publishing a separate study that analyzes potential adjustment factors that should be applied to certain types of hospitals when comparing them to non-pure peers using a price-sensitive efficiency index.
Third, we recommend attributing episodes only to providers who have a substantial impact on the episode of care. We suggest a threshold of at least 25% of total professional costs, and believe that it would be acceptable to attribute cases to multiple providers if they each had a substantial impact on the episode of care. We offer some evidence in support of that threshold and methodology.
Finally, we recommend that provider performance reporting should distinguish between differences in utilization and cost per unit. All performance should be reported in valid statistical groupings to reflect the relative performance of the provider, avoiding strict numerical rankings where the risk of misclassification is high. Generally, reporting performance on efficiency should be linked to reporting performance on quality to better understand, measure and communicate the value that is delivered by physicians and hospitals.
We recognize that not all organizations will, or can, apply all the recommendations listed in this paper. We also recognize that the science, experimentation and research on measuring efficiency should continue in earnest in an open learning community. To that end, we recommend that any organization measuring provider efficiency (1) clearly communicate to all stakeholders (in particular providers and purchasers) the specific methodology used in arriving at the results and any rationale for varying from the recommendations in this paper, (2) publish the confidence interval around the results, and (3) participate in an on-line learning forum at www.regence.com/research to share the results of their work and advance the science in this field.
We also greatly encourage the developers of models and methodologies that measure provider efficiency to make their models available to researchers at very low or no cost, and to develop "freeware" versions of their products that can be used by providers and others to help improve their performance.