Current public reporting programs

Given the rapid spread of PR initiatives, it is beyond the scope of this srticle to describe the full range of programs currently underway around the nation. To give a sense of what some providers face, however, we outline the ongoing PR activities operating in a single state, California. Although California may have more initiatives than some other states, the programs there likely reflect the direction the nation is heading and can help providers everywhere understand what they may face at some point in the future.

PR programs in California were originally initiated by the state government through its Office of Statewide Health Planning and Development, which first started reporting publicly hospital-specific, risk-adjusted mortality rates for various conditions (myocardial infarction was the first) in the 1990s. Since then, other state regulatory agencies, employer groups, multi-stakeholder collaboratives, and even providers themselves have adopted PR. For example, efforts to create incentive-based quality improvement programs for California physicians have been led by the IHA. The IHA is a multi-stakeholder coalition whose membership includes major health plans, physician groups, and hospital systems, which work together with academic, consumer, purchaser, pharmaceutical, and health care technology representatives. Among the group’s principal projects are performance assessment and PR programs. Table 1 summarizes current PR projects supported by IHA members and other major PR activities ongoing in California. California providers are also subject to national PR initiatives such as PR about hospital performance by the Joint Commission on Accreditation of Health care Organization (JCAHO).

Current pay-for-performance programs

The Leapfrog Group, a coalition of large employers that seeks to improve the quality of health care and collectively covers approximately 40 million beneficiaries, maintains a compendium of PFP programs on its Web site. In 2006, the compendium included more than 100 PFP programs nationwide. In the United States, more than half the health maintenance organizations in the private sector, a group that collectively has more than 80% of health maintenance organization enrollees, have initiated such programs.

Although still in early phases, the US Congress recently mandated the Center for Medicare and Medicaid Services (CMS) to develop a plan to introduce PFP programs into Medicare. To address this, CMS has engaged in several demonstration projects. In the Premier hospital quality improvement demonstration project, CMS has partnered with Premier, a national hospital performance improvement alliance based in North Carolina whose participants comprise 1700 not-for-profit hospitals and health systems with the core purpose “to improve the health of communities.” CMS and Premier have adopted a set of performance indicators based on validated measures adapted from the American Hospital Association, the National Quality Forum, JCAHO, the Leapfrog Group and the federal Agency for Healthcare Research and Quality.

Table 2 lists the process and outcome measures being used in the CMS Premier demonstration project, such as measures for patients with myocardial infarction, heart failure, pneumonia, coronary artery bypass graft, hip, and knee replacement surgeries. CMS calculates annual composite quality scores for each clinical condition for each hospital by combining performance scores for each individual measure within the clinical area. Hospitals in the top 10% for a given clinical area receive a bonus of 2% of their Medicare payments for the measured condition, whereas hospitals in the second decile are paid a 1% bonus. In the third year of the demonstration project, hospitals that fail to score above the levels that represented the thresholds for ninth and tenth deciles of performance in the first year of the program suffer 1% and 2% lower diagnostic related group (DRG) payments, respectively. By using the thresholds defined in the first year of the program for the penalty portion of the demonstration, CMS gave all hospitals the opportunity to improve above a known target and avoid creating a situation in which 20% of participating hospitals would have been guaranteed a penalty (which would have happened if they had used current year performance).

Current pay-for-performance programs

The Leapfrog Group, a coalition of large employers that seeks to improve the quality of health care and collectively covers approximately 40 million beneficiaries, maintains a compendium of PFP programs on its Web site. In 2006, the compendium included more than 100 PFP programs nationwide. In the United States, more than half the health maintenance organizations in the private sector, a group that collectively has more than 80% of health maintenance organization enrollees, have initiated such programs.

Although still in early phases, the US Congress recently mandated the Center for Medicare and Medicaid Services (CMS) to develop a plan to introduce PFP programs into Medicare. To address this, CMS has engaged in several demonstration projects. In the Premier hospital quality improvement demonstration project, CMS has partnered with Premier, a national hospital performance improvement alliance based in North Carolina whose participants comprise 1700 not-for-profit hospitals and health systems with the core purpose “to improve the health of communities.” CMS and Premier have adopted a set of performance indicators based on validated measures adapted from the American Hospital Association, the National Quality Forum, JCAHO, the Leapfrog Group and the federal Agency for Healthcare Research and Quality.

Table 2 lists the process and outcome measures being used in the CMS Premier demonstration project, such as measures for patients with myocardial infarction, heart failure, pneumonia, coronary artery bypass graft, hip, and knee replacement surgeries. CMS calculates annual composite quality scores for each clinical condition for each hospital by combining performance scores for each individual measure within the clinical area. Hospitals in the top 10% for a given clinical area receive a bonus of 2% of their Medicare payments for the measured condition, whereas hospitals in the second decile are paid a 1% bonus. In the third year of the demonstration project, hospitals that fail to score above the levels that represented the thresholds for ninth and tenth deciles of performance in the first year of the program suffer 1% and 2% lower diagnostic related group (DRG) payments, respectively. By using the thresholds defined in the first year of the program for the penalty portion of the demonstration, CMS gave all hospitals the opportunity to improve above a known target and avoid creating a situation in which 20% of participating hospitals would have been guaranteed a penalty (which would have happened if they had used current year performance).

Assessing quality of care: key concepts

Whether PR and PFP improve quality of care is not yet certain. Most of what we know about how people respond to incentive-based programs has emerged from studies conducted in other fields, such as economics, psychology, and organizational behavior. The theoretic frameworks developed within these fields are not necessarily directly applicable to health care, partly because of the complex web of relationships among physicians, patients, third-party payers, and purchasers of health insurance, which makes it difficult to study, let alone confidently identify, the key factors influencing provider and consumer responses to PR or PFP. We do know that although PR and PFP may create incentives, they do so in a milieu rich with myriad existing financial incentives. On one hand, traditional fee-for-service insurance creates incentives to provide more services, while on the other hand, capitation creates incentives to provide fewer services. These financial factors interplay with powerful nonfinancial motivations, such as professionalism, the desire to protect one’s reputation among peers, and altruism. In this setting, it would be unwise to assume that new incentive programs, such as PR or PFP, will become the dominant determinants of behavior or will have any effect at all.

Further complicating efforts to understand the impact of PR and PFP is the fact that the gold standard of research design, the randomized controlled trial, is rarely feasible for PR or PFP. It is unlikely that researchers would have the wherewithal to randomize communities to have a PR program in one arm and no PR program in the control arm. It is possible to observe whether consumers’ choice of provider changes after reports about quality of care are released or whether providers’ quality improvement efforts change after the introduction of PR. In terms of PFP, in some cases, providers can be randomized in PFP trials. In other cases, natural experiments have occurred; such as when a health plan that provides insurance in two different states introduces PFP in one state but not in the other. Before exploring the available evidence about PR and PFP, it may be useful to describe some key concepts in performance measurement.

Quality improvement efforts in other industries, and more recently in health care, have typically addressed three domains: structural, process, and outcome quality. In 1966, Donabedian applied this framework to health care. In the health care context, structural quality refers to the resources available to deliver care—including equipment and human resources—and the physical environment in which care is delivered. Some examples of structural measures of care that have received significant attention recently are hospital and surgeon annual patient volume (on the assumption that higher volume translates to better quality) and availability of computerized physician order entry for inpatients (on the assumption that it will reduce medication errors). Process measures assess whether the appropriate care was delivered. An example would be to measure whether prophylactic antibiotics are given preoperatively and discontinued within a timely fashion. Finally, outcome quality is determined by changes in health status and is described as the end pathway of structural and process quality. In this theoretic framework, poor process and structural quality should result in worse outcomes.

In general, outcome measures need to be risk adjusted, correcting an observed outcome rate to account for the severity of illness of each provider’s or hospital’s patient population. In general, process and structural measures do not require such adjustment. For almost all process measures, decision rules need to be developed to ensure that performance assessments only include patients for whom the measured process of care is truly appropriate.