- CMS should not move forward with updating its 2017 Hospital Quality Star Ratings and should scrap the whole system from the Hospital Compare website, according to the American Hospital Association.
In a letter from AHA’s Senior Vice President of Public Policy Analysis and Development Ashley B. Thompson, the organization claimed that the CMS system is not an effective way to educate patients and families on hospital quality.
“The AHA has long supported transparency and continues to share CMS’s goal of making the data on Hospital Compare easier for consumers to understand,” stated the letter addressed to CMS CMO Kate Goodrich, MD. “However, CMS’s flawed approach to star ratings undermines this goal by providing an inaccurate, misleading picture of hospital quality.”
AHA has long advocated against the CMS star ratings’ inclusion on the Hospital Compare website, contending that the five-star rating does not capture the nuance of hospital quality. Additionally, the organization has critiqued CMS’s methodology for calculating the star ratings, stating that it has resulted in erroneous ratings for some hospitals.
Thompson conceded that the proposed updates for the star ratings corrects some of those methodology errors. The updates, which were published in August 2017 under contract with the Yale New Haven Hospital System Center for Outcomes Research and Evaluation (YNHHS CORE), focus on three areas:
- K-means clustering using complete convergence
- Hospital summary score winsorization
- Resequencing reporting thresholds
K-means clustering concerns to the way in which CMS assigns hospitals into their star rating category. Ideally, hospitals within one category will have similar summary scores, and those in other categories will have different summary scores. This update will include multiple iterations of k-means clustering, ensuring more accurate categorization.
Summary score winsorization refers to the process by which CMS eliminates outlier hospitals, thus making it easier to distribute hospitals during the k-means clustering process. Because of proposed updates to the k-means clustering process, CMS contends that winsorization may not be necessary.
Resequencing reporting thresholds means that CMS would reverse the order in which it applies public reporting thresholds. Traditionally, the agency has applied public reporting thresholds after assigning hospitals to star rating groups.
The proposal calls for CMS to apply public reporting thresholds before assigning hospitals to their quality groups. This would allow only hospitals that meet public reporting thresholds to be categorized into star rating groups.
Although some of these proposed changes could correct the star ratings methodology pitfalls, AHA argued that there are still numerous improvements to be made.
“Even if CMS can improve the implementation of the current methodology, we continue to have significant concerns about the conceptual underpinnings of star ratings,” Thompson said. “The measures included in the ratings were never intended to create a single, representative score of hospital quality. Furthermore, the ratings often do not reflect the aspects of care most relevant to a particular patient’s needs.”
AHA touched on areas outside of the main three points in the CMS proposal, including the latent variable measure (LVM). The LVM calculates an average quality score for each of the star ratings groups, assuming that there is a latent quality characteristic amongst hospitals within a certain rating group.
Traditionally, CMS has included all quality measures when calculating the LVM. AHA argued that those individual hospital traits that are negatively associated within a quality group – meaning an individual hospital underperforms in this area – be excluded from the LVM, lest some hospitals be unduly penalized or rewarded.
“CMS suggests that the impact of the negative factor loadings on the overall score is relatively modest, which we believe is a strong argument for removing them from the star rating system,” AHA pointed out. “If the measures do not have much impact, and the impact they do have is the opposite of what was intended, then there is no reason to keep the measures in the star rating.”
The weighting methodology is also flawed, AHA asserted. CMS weights different quality measurements differently based upon importance to quality care.
“It is unclear to what extent CMS’s choices were informed by any systematic assessment of patients and family preferences,” AHA explained. “To implement a less arbitrary and more patient-centered approach, we recommend that the agency survey patients and families to obtain a statistically significant sampling of views about how to weight the measure groups.”
AHA agreed with CMS’s proposal to exclude hospitals that do not meet public reporting thresholds because those hospitals skew the quality groups. Additionally, AHA agreed with the proposal to run multiple k-means clusters, stating that this will ensure that hospitals land in the appropriate groups.
The issues AHA noted are key to ensuring that patients and families can make informed decisions about where to receive medical treatment. Including erroneous star ratings is a detriment to patients, the organization argued.
Although AHA agrees with the federal agency’s intention of offering transparency in hospital quality for patients and families, it does not feel that the star ratings currently hold value for patients and families. Reformulating these star ratings will improve the process for patients.
“At a minimum, the AHA strongly urges CMS to remove the star ratings from Hospital Compare and not republish them until it corrects the errors and outside experts agree that the updated methodology is executed correctly,” the organization concluded.