All Content

Accurate Measures?


For SHM’s official position on issues like healthcare reform, value-based purchasing, and medical errors, visit

Everyone’s talking about quality. Encouraging high-value care is one of the stated objectives of the value-based purchasing program being rolled out by the Centers for Medicare & Medicaid Services (CMS). It’s also the subject of a new report to Congress from the Department of Health and Human Services (HHS), “National Strategy for Quality Improvement in Health Care” ( For its part, SHM is placing added emphasis on a range of mentored quality-improvement (QI) initiatives for hospitalists.

Amid the flurry of activity, researchers are still attempting to address a central question that could determine the success or failure of many such efforts: How do you accurately measure what constitutes high-quality care?

Chris Murray, MD, DPhil, director of the Seattle-based Institute for Health Metrics and Evaluation, says the healthcare field traditionally has tried to assess quality in three main ways. One is to ask patients about their own experience: Were they satisfied with the level of care they received? Another is to assess what are known as process of care measures: Did the providers follow guidelines in providing patients with appropriate care? The third is to look at risk-adjusted outcomes: How did the patients ultimately fare?

Focused on Facts

CMS’s value-based purchasing program, at least initially, is focusing on the first two types of metrics. Process measures, Dr. Murray says, are popular in part because they’re relatively easy to gauge. For many of them, however, “the connection to improved health is a bit weak,” he says. Whether heart patients get a prescription for a beta-blocker drug, for example, doesn’t address the outcome. “The problem there is that we don’t know if they ever filled the prescription or if the patient takes the beta-blocker,” Dr. Murray says.

The world is getting better at predicting rehospitalization, predicting death from attributes of the patient. If you can do a better job at risk adjustment, you can do a better job on identifying quality.—Chris Murray, MD, DPhil, director, Institute for Health Metrics and Evaluation, Seattle

That uncertainty feeds into the larger question of how broadly to consider the accountability of providers when measuring quality. “Should we be thinking that quality means putting in place the supports required for a patient to actually achieve a good outcome, or just offering them?” Dr. Murray asks. The debate might be far from settled, but a growing number of tools and studies are at least helping researchers to connect the dots on how care is delivered, on what kind of practices might affect outcomes the most, and how a community’s underlying risks could influence both considerations.

A recent Annals of Internal Medicine study that scrutinized 30-day mortality rates for heart-attack patients found few quantitative differences between the top 5% and bottom 5% of hospitals, based on rates published on the CMS Hospital Compare website.1 Site visits and in-depth interviews with nearly 160 medical staff members, however, uncovered some telling distinctions.

The study found that following evidence-based protocols and processes, while important, likely is not sufficient to attain a high performance level in caring for heart-attack patients. Instead, “high-performing hospitals were characterized by an organizational culture that supported efforts to improve AMI [acute myocardial infarction] care across the hospital.” In other words, everyone from management to the medical staff was fully invested in QI efforts. Notably, the staff “reported the presence of physician champions and empowered nursing staff, pharmacist involvement in patient care, and high qualification standards for all staff.”

For its 13th annual HealthGrades Quality in America study, the Denver-based ratings organization HealthGrades tried to look more quantitatively at the link between top hospitals and patient outcomes. Its study coauthors culled data from roughly 40 million Medicare discharges from 2007 through 2009 for most of the nation’s 5,000 hospitals, and assigned ratings based on 26 measures related to mortality and complication rates (

If all hospitals were performing on par with what HealthGrades terms a five-star hospital, the study suggests the U.S. healthcare system could have saved the lives of more than 230,000 Medicare beneficiaries over the three-year period. More than half of the preventable deaths were associated with sepsis, pneumonia, respiratory failure, and heart failure.

Although the high number raises the question of whether some preventable deaths might exist only on paper, the study does raise other eye-popping calculations. Typical patients who went to a five-star hospital instead of a one-star hospital had a 72% lower risk of dying and reduced their risk by 53% compared with U.S. hospitals overall. The survival advantage persisted after hospitalization, too: Patients discharged from five-star-rated hospitals were 57% less likely to die within 30 days than all patients.

Ali Mokdad, PhD, professor of global health at the Institute for Health Metrics and Evaluation, says one big caveat to such rankings is the matter of adjusted risk. What kind of patient populations are these hospitals treating? Are people in the area inherently less healthy? Are significant barriers to healthcare blocking access to preventive medicine?

Dr. Murray says measuring quality with risk-adjusted outcomes has periodically fallen in and out of favor, due in part to concerns over how the risk is calculated and whether the assessments could be biased against providers that see more difficult patients. Nonetheless, he believes the metric is underused in the U.S. “I think the pendulum went way away from risk-adjusted outcomes to process measures too much, and we need to have a mixed combination,” he says.

With improvements to the methodology, he sees a wealth of potential in picking out risk predictors from large data sets. “The world is getting better at predicting rehospitalization, predicting death from attributes of the patient,” he says. “If you can do a better job at risk adjustment, you can do a better job on identifying quality.”

Risk Adjustment

One area in which the U.S. has lagged is in integrating the risk of death due to chronic conditions into broader measures of healthcare. At the recent Global Health Metrics & Evaluation Conference in Seattle, Dr. Mokdad pointed out the stringent oversight applied to commercial airliners. An avoidable crash and loss of life would quickly lead to a full-scale investigation. Why, he wondered, can’t the same scrutiny be brought to bear on preventable deaths due to chronic conditions such as diabetes and heart disease?

An ambitious new surveillance project, in fact, is trying to do exactly that. Known as the Monitoring Disparities in Chronic Conditions (MDCC) Study, the effort will use Washington state’s King County as a test case to hone the necessary data collection techniques. If it pans out, the study could become a national model for how to assess a population’s health status. “You know how a physician takes your pulse?” Dr. Mokdad says. “We’re doing that for the community.”

The research team, which includes Dr. Mokdad, Dr. Murray, and collaborators from Dartmouth and Harvard universities, will administer in-depth, culturally sensitive surveys to more than 3,000 county residents. A subset of 750 participants also will receive physical exams that measure markers of health and activity.

Send Us a Letter

Want to share your opinion or help educate colleagues on a new protocol? Email your “Letter to the Editor” of no more than 750 words to Jason Carris at

One goal is to work out how to efficiently integrate data from multiple sources so researchers can apply their risk adjustment strategies. For example, can they get enough information to ask how many heart attack patients are on beta-blockers one year after a hospital discharge? “There is also this big question of community background health risk,” Dr. Murray says. “Is this a community where people are just sicker, and how do you factor that in addition to taking into account the comorbidities that individuals have when they show up in the hospital?”

Researchers are close to obtaining enough information on such key factors as blood pressure, cholesterol, tobacco use, and obesity to actually rate communities according to risk, he says.2 “That’s never been done at the local level, and I think it’s where we need to go to truly put things on a level playing field when you’re assessing quality.” TH

Bryn Nelson is a freelance medical writer based in Seattle.


  1. Curry LA, Spatz E, Cherlin E, et al. What distinguishes top-performing hospitals in acute myocardial infarction mortality rates? Ann Intern Med. 2001;154(6):384-390.
  2. Murray CJ, Kulkarni SC, Michaud C, et al. Eight Americas: investigating mortality disparities across races, counties, and race-counties in the United States. PLoS Med. 2006;3(9):e260.

Next Article:

   Comments ()