CMS Hospital Quality Star Ratings Update

We continue to monitor the CMS hospital quality star ratings on its Hospital Compare website. The overall star ratings are based on 64 quality measures grouped under three process categories—effectiveness of care, efficient use of medical imaging, and timeliness of care—and four outcomes categories: mortality, patient experience, readmissions, and safety of care. Few hospitals, even among the nation’s best, receive a five-star rating.

Recent Data

According to recent CMS star-rating data as reported by the Advisory Board:

Out of 3,629 hospitals eligible for a rating:

83 hospitals received a five-star rating;

946 received a four-star rating;

1,794 received a three-star rating;

694 received a two-star rating; and

112 received a one-star rating.

CMS did not assign star ratings to 969 hospitals for which it lacked sufficient data.

This clustering around the median rating is similar to what we have seen with other programs like the value-modifier. Does CMS create algorithms to create this distribution? A slight edge toward the bottom for the “underperforming” groups, and a small amount who “win” at the top? Something we will monitor going forward.

Ratings continue to unfairly punish “poor” hospitals

As we wrote earlier this year, the star rating system rewards hospitals that serve mostly affluent patients and punishes those serving the poor. Research by Bloomberg BNA compares star ratings of hospitals, indicating a correlation between high star ratings and high household income, and a corresponding correlation between low ratings and low income. Critics of the rating system point out that low-income patients are more likely to have difficulty accessing transportation for both routine primary care and post-discharge follow-up care. There are also consistently low ratings of academic medical centers, which are generally considered among the nation’s best hospitals and which are often located in low income urban areas. 

Data reliability questioned in different rating system

As has been reported, in the search for hospital quality measures that matter, all kinds of rating systems are coming under scrutiny, including those used by the Leapfrog Group, a nonprofit advocate of quality and safety in healthcare.

A study from the University of Michigan concludes that the group's Safe Practice Score (SPS) produces different results than those used by Medicare's Hospital Compare to track common complications and readmissions.

The Leapfrog findings "skew toward positive self-report[ing]," according to the study, which appeared in the journal Health and was entitled "Dissecting Leapfrog."

Plenty of problems with the CMS system, too

As Health Affairs reported last year, the structure of the CMS program has problems, too. They note that a single score to describe hospital quality is probably not useful for consumer. “Given that the quality for different types of care can vary widely within a single institution, it is unlikely that a single summary score would accurately represent the quality of care for all conditions or procedures at one hospital.

To construct the summary star scores, some fairly complex statistical calculations are performed, which essentially use rank order performance on individual measures, weighted by importance to come up with a summary score. The end result is a distribution of summary scores that approximates a bell-shaped curve with 48 percent of hospitals assigned 3-stars; about 3 percent assigned each 1- and 5-stars, and the rest 2- or 4-stars.”

But there are many problems with using a curve. Health Affairs continues: “First, it implies a meaningful difference in performance when there might not be one. For many of the individual measures from which the summary score is derived most hospitals are no different than the national average. Second, it implies that many stars equal high quality and few stars low. Regardless of whether quality across hospitals is uniformly high, low, or average, the curve will distribute hospitals across the 5-stars. Consider the measures reported in the ‘effectiveness of care’ domain. The average national score is over 92 percent for most of the measures; for several it approaches 100 percent. There is little clinically meaningful difference in scores across hospitals and the performance is uniformly high.”

Comments

[Thomas Sullivan]

That is a good question, data typically is over 1 year old so data published in 2017 would be from the 2015 performance year.

I recommend readers to spend some time on the hospital compare website. This year, I gave a talk on quality improvement at a community hospital, the staff thought their hospital was a 4 star hospital when on hopsital compare it was a 2 star hospital. Many Academic medical centers are less than 3 stars.

https://www.medicare.gov/hospitalcompare/Data/Data-Updated.html#

[Bonnie Freudenberg]

What is the performance period for the October 2017 report?
Thank you.
Bonnie F.