fah hospital policy blog

Perspectives on health policy affecting America’s hospitals and the patients we serve.

Closer Look: Flawed Methodology Means Hospital Star Ratings Could Mislead Patients

July 28, 2016 | FAH Policy Blog Team

Category: FAH News, Health Care Delivery, Quality

The Federation of American Hospitals is disappointed by the release of the hospital overall star ratings. We have joined with our hospital partners several times to express concerns about the statistical methodology used by CMS to create these stars. FAH believes that hospitals and the public deserve an accurate portrayal of a facility’s performance and that CMS should undertake additional review and adjustments to its star rating methodology.

(Read FAH President and CEO Chip Kahn’s statement here)

The process of creating overall hospital star ratings is complicated and requires the melding and narrowing of 62 complex data points into a five star matrix.

For example, the overall star rating program is a depiction of common care measures reported on Hospital Compare.  The Stars display 7 categories of care, and a hospital must have at least three measures in three categories to be included in the overall star ratings.  Some hospitals may not treat enough patients to be eligible to receive an overall rating.  Some hospitals displayed with stars will have as few as 9 measures (3 measures in 3 categories) making up their star rating, while other hospitals may have as many as 62 measures across all 7 categories.  Each category score is weighted and then combined to create the overall star rating.  It is important to note that the methodology does not adjust for patient characteristics that can influence health care outcomes outside a hospital’s control, such as socioeconomic status or sociodemographic factors.  In addition, more than 20 percent of hospitals will not receive an overall star rating because they did not treat enough patients in at least three categories of care to generate a star.  In other words, the more measures used to determine a hospital’s score (a proxy for the comprehensiveness of care delivered) and, arguably, community access to care, the lower the star rating, while hospitals with fewer measures achieve higher star ratings. 

In extensive communication with the Administration (including letters in March 2016 and July 2016), the FAH and other national hospital associations highlighted these flaws and asked the agency to share additional information and also to reconsider the methodology used to create the star ratings.  A bipartisan majority in both houses of Congress (60 Senators and 225 House Members) have also signed letters to CMS asking for this process to be fixed.  Unfortunately, while some additional information was shared, the agency chose not to re-evaluate their statistical methodology to address the unintended adverse consequences of the star methodology.

FAH strongly supports transparency of the quality of care hospitals provide, however we continue to have significant concerns about the current star ratings methodology. FAH encourage CMS to undertake more robust analysis of its own data and reconsider the risk adjustment and statistical factors used to aggregate category scores.

We will continue to work with our health care partners and policy makers to ensure that the star rating system is constructed properly and that the stars provide accurate information about the quality of care which patients and their families can factor into their decision making about their choice of hospital..

We support the efforts of Reps. Jim Renacci (R-OH) and Kathleen Rice (D-NY), who recently introduced the Hospital Quality Rating Transparency Act of 2016. This legislation would require CMS to accept comments for 60 days on its methodology and the data used to construct the stars. In addition, it asks that the methodology and data be validated by an independent third party. The bill also would require the agency to remove the star ratings from the Hospital Compare website if they are posted prior to the bill’s enactment.

Choosing a health care provider is a complex and personal decision that reflects an individual’s preferences and needs.  Patients and their families should consider many factors when choosing a hospital, including advice from their physician, their personal care plan along with safety and quality information.  FAH believes that a star rating system, properly constructed, can be a trusted, value-added piece of that process.  Regrettably, we are not there yet.  The hospital star ratings CMS released reveal more about the limits of the methodology CMS applied than they do the quality of care hospitals provide.  We need to get this right or run the risk of misleading patients on such an important decision as choosing where to get their care.