Beware Hospital Compare? New Measures Highlight Questions Surrounding Healthcare Quality Report Cards

Anne-Marie J. Audet, MD, MSc, SM To get where we want to go in American healthcare, we need a more thoroughly supported measure development infrastructure.

—Anne-Marie J. Audet, MD, MSc, SM, vice president, Health System Quality and Efficiency, Commonwealth Fund

The Centers for Medicare & Medicaid Services (CMS) has been publicly reporting performance measures on its Hospital Compare website ( since 2005, focusing on processes of care, patient outcomes, patient satisfaction, patient safety, and other measures. A recent addition of patient-safety metrics has rekindled skeptical questions about the validity, purpose, and effectiveness of public healthcare quality report cards, while highlighting the need for hospitalists and their institutions to remain vigilant in the struggle to ensure that they are compared and rewarded fairly and appropriately.

Provocative Measures

Last fall, CMS began posting “Serious Complications and Deaths” measures, developed by the Agency for Healthcare Research and Quality (AHRQ). The measures score individual hospitals according to the rates at which their patients suffer from:

  • Pneumothorax due to medical treatment;
  • Post-operative VTE;
  • Post-operative abdominal or pelvic dehiscence; and
  • Accidental lacerations from medical treatment.

Four other serious complication measures (pressure ulcers, catheter and bloodstream infections, and hip fractures from falling after surgery) are folded into a separate composite score for each hospital, while another composite score for “Deaths for Certain Conditions” is based on a hospital’s post-admission mortality rate for hip fractures, acute MI, heart failure, stroke, GI bleed, and pneumonia.

National and local media reports have thrust these dramatic metrics into the public eye, putting many hospitals on the spot to explain their putative breaches of patient safety. A closer inspection of the metrics, however, reveals plausible criticisms of their shortcomings.

Methodological Weakness

The new metrics are derived from Medicare claims data instead of medical chart abstractions, which experts say weakens their validity significantly and makes their use for provider profiling questionable. Moreover, claims data are based on records that were never designed to capture the sort of clinical nuances needed for valid and equitable risk adjustment (see “Methodological Challenges to Quality Metrics,” below). “Serious Complications and Deaths” rates based on these data, critics maintain, lack validity for meaningful hospital comparisons because they can exaggerate problems at hospitals that treat a high volume of complicated patients and use more invasive procedures to do so, such as teaching hospitals in academic medical centers.1

The ante gets upped when CMS eventually begins adding patient-safety measures to the Hospital Value-Based Purchasing (HVBP) program, which rewards or punishes hospitals financially, depending on their performance on the metrics. CMS is considering adding the Serious Complications and Deaths measures to the HVPB program in the near future.

Leave a Reply

Your email address will not be published. Required fields are marked *