Executive Summary
Established in 1999, The Commonwealth Fund's International Working Group on Quality Indicators represents a critical step forward in the development of international measures of health care quality. This unique collaboration and technical exchange brought together representatives of five industrialized countries—Australia, Canada, New Zealand, the United Kingdom, and the United States—committed to the development of a set of indicators to help benchmark and compare health care system performance across countries, while helping clinical leaders and policymakers in each country identify areas for improvement. In addition to government officials, Working Group members included leading academic experts in quality measurement, representatives from the Organization for Economic Cooperation and Development (OECD), the World Health Organization (WHO), The Nuffield Trust, the Canadian Council on Health Services Accreditation, and The Commonwealth Fund.Underpinning the Working Group's activities and the findings presented in this report was a multistep process to systematically identify measures of quality that could be used to compare performance across countries. This process included: mapping the conceptual domains of quality; comparing the national quality frameworks used by each country; cataloguing the available indicators in each domain; adopting criteria for the selection of a set of international quality indicators; assessing and selecting indicators that met the criteria; and collecting data for the initial indicator set.
Using the Canadian Institute for Health Improvement Performance Framework as the organizing construct for defining the domains of quality, the Working Group focused its initial efforts on five subdomains of health system performance: effectiveness, appropriateness, accessibility, continuity, and acceptability. Starting with over 1,000 potential indicators that were currently available at the national or regional level in one or more countries, the Working Group selected an initial set of indicators based on agreed criteria, which required that the indicator be meaningful, important, and actionable for policymakers; scientifically sound; comparable internationally; and feasible to report.
The initial results represent great progress in international quality measurement. For the five countries, the Working Group has produced performance data on 40 quality indicators, including five-year survival rates for breast, cervical, and colorectal cancers, childhood leukemia and non-Hodgkin's lymphoma, and kidney and liver transplants; 30-day case-fatality rates following the incidence of heart attack and stroke; asthma mortality rates; suicide rates; breast and cervical cancer screening rates; vaccination rates; smoking rates; waiting times for primary, emergency, and specialty care and elective surgery; measures of patient–doctor communication and coordination of care; and indicators of financial barriers to care.
The results show that no country consistently scored the best or worst on all of the indicators; each country had either the best or worst score on at least one indicator. In addition, each country has at least one area of care where it could potentially learn from international experience. The key findings of this report, presented by country, follow below.
Australia
Areas of good performance: Cancer survival rates were generally high (excepting childhood leukemia). Rates were highest for cervical cancer and non- Hodgkin's lymphoma; breast and cervical cancer screening rates were high as well. Asthma mortality was relatively low. Influenza and polio vaccination rates were high. Ratings of access to care and physician responsiveness were high.
Opportunities for improvement: The incidence of pertussis (whooping cough) was much higher than in the four other countries.
Canada
Areas of good performance: Cancer survival rates were generally average or above average and were highest for childhood leukemia. Transplant survival rates were highest. Canadians reported very few financial barriers to getting medical care, diagnostic tests, or prescription drugs.
Opportunities for improvement: Acute myocardial infarction (heart attack) case-fatality was higher in Canada than in Australia or New Zealand in older age groups. Pertussis incidence was much higher than in New Zealand, the U.K., or the U.S. Canadians reported difficulty seeing a specialist, getting care on nights and weekends, and getting same-day doctor appointments when needed.
New Zealand
Areas of good performance: The improvement in asthma mortality over the past 20 years is a true success story, although some room for further improvement may exist. The relative survival rate for colorectal cancer was the highest of the five countries. New Zealanders reported the fewest problems accessing care on nights and weekends, getting same-day appointments, and waiting for emergency care. They also reported the fewest coordination-of-care problems, good patient–doctor communication, and the highest overall physician responsiveness.
Opportunities for improvement: The suicide rate in New Zealand, particularly among younger people, is much higher than in the other four countries. Stroke case-fatality rates were higher among older age groups. Influenza and polio vaccination rates were relatively low. Breast cancer screening rates were lowest in New Zealand.
United Kingdom
Areas of good performance: Suicide rates were notably lower in England* than in the other four countries. The polio vaccination rate was the highest. The incidence of pertussis was the lowest. U.K. citizens reported virtually no financial barriers to medical care, diagnostic tests, or prescription drugs and the least difficulty seeing a specialist.
Opportunities for improvement: Cancer survival rates were lowest. Measles incidence was higher than elsewhere. U.K. citizens reported the longest waits for elective surgery. U.K. physicians were rated poorly on asking patients for their opinion, discussing the emotional burden of illness, and overall responsiveness.
United States
Areas of good performance: Breast cancer survival rates were highest in the U.S. Cervical cancer screening rates were very high. Waiting times for elective surgery were lowest. U.S. doctors were the most likely to ask for the patient's opinion and to discuss the emotional burden of illness.
Opportunities for improvement: Asthma mortality rates are increasing in the United States while they are decreasing in the other countries. Transplant survival rates were relatively low. U.S. citizens reported trouble seeing doctors, particularly on nights and weekends and for same-day appointments. They also reported the most financial barriers to care and the most coordination-of-care problems.
It should be noted that the initial list of 40 quality indicators presented in this report, distilled from a compendium of more than 1,000 indicators, is opportunistic rather than comprehensive. There are significant gaps in the domains covered, with many conditions that account for a major share of the burden of disease—such as heart disease, mental health, and diabetes—barely covered. High-volume procedures in obstetrics and orthopedics and high-cost interventions, such as new pharmaceuticals, are not covered at all. The lack of available indicators in so many areas indicates the magnitude of work still to be done to develop robust data sets that can adequately measure the processes and outcomes of health care. Nonetheless, the initial list, while lacking comprehensiveness, is an important starting point for comparing different aspects of health care quality in the five countries and prompting questions about how both the data and performance might be improved. * Some indicators represent England and some represent the entire United Kingdom.
Recommendations
The International Working Group on Quality Indicators recommends that this first set of international quality indicators be used to:
- draw attention to potential opportunities for improving the quality of health care in the five countries;
- raise questions about why some countries do well on some measures and others do poorly;
- provoke debate within countries about health care priorities and policies; and
- stimulate efforts to reexamine, refine, and improve the data that have been presented and to encourage further commitment and resources to improving the availability of health care quality data in all our countries.
Building on the work of The Commonwealth Fund's International Working Group on Quality Indicators and a similar effort by five Scandinavian countries under the auspices of the Nordic Council, the OECD initiated the International Healthcare Quality Indicators Project in January 2003. Under this project, the OECD aims to take this work forward by expanding the number of countries involved, institutionalizing the collection of these indicators, and developing additional quality indicators to provide the scope and depth of measures needed to judge performance across health care systems.
* Some indicators represent England and some represent the entire United Kingdom.