What Is Non-Response Bias in Patient Satisfaction, CAHPS Surveys?

February 05, 2019 - Patient satisfaction surveys and CAHPS scores are essential metrics that healthcare organizations use to understand their patients’ experiences of care and their next practice improvement moves. But surveying the whole patient population can be difficult, leading to non-response bias and skewed survey results.

Non-response bias, or participation bias, is the influence that the respondents to a certain survey may have because of their personal qualities and demographic characteristics. When only a certain group of people participates in a survey, administrators miss out on a broad and diverse set of individuals’ opinions, needs, or experiences.

In the healthcare and patient satisfaction space, this means that organization leaders only learn about the preferences of one or two specific populations.

For example, if only English-speaking patients responded to a patient satisfaction survey, the survey is only useful for understanding the preferences and experiences of English-speaking patients. There is no way for organization leaders to examine the experiences of those who do not speak English.

This poses a considerable issue, especially because practices use survey responses to inform practice improvement plans. How can a hospital ensure an initiative will benefit all patients if only those who speak English are surveyed? How can they understand the efficacy of a digital patient payment system if only younger, more native technology users respond to the survey?

READ MORE: How Hospitals Can Raise Patient Satisfaction, CAHPS Scores

Below, PatientEngagementHIT.com explores the concept of non-response bias in patient satisfaction and CAHPS (Consumer Assessment of Healthcare Processes and Systems) survey scores and how to prevent that participation bias.

How much does participation bias impact patient satisfaction surveys?

Participation bias is a key consideration for all statisticians. It is an important limitation they must all examine before drawing conclusions. But to what extent does non-response bias impact surveying in healthcare?

Data suggests several key patient populations are excluded from patient satisfaction surveys, most of which are vulnerable or otherwise underserved.

A 2011 study published in PLOS One examined the specific groups that tend not to respond to patient satisfaction surveys. Patients with substance use disorder (SUD), language barriers, cognitive impairments, and partial or total blindness are less likely to respond to patient satisfaction and CAHPS surveys than those with fewer needs.

A separate study published in BMC Health Services Research found similar results. Older patients, males, and those with public insurance faced more barriers in responding to patient satisfaction surveys than their younger, female, and privately insured counterparts.

Patients who fit these demographic criteria tend to face more social determinants of health, or social factors that hamper their interactions with the healthcare industry. These patients may not understand the questions asked in a patient satisfaction survey or lack the self-efficacy and activation to answer a survey.

To be clear, demographics are not the sole factor that influences whether a patient does or does not respond to a satisfaction survey. Some older, publicly insured males may answer a patient satisfaction survey, even though the data suggests they are unlikely to.

Personal characteristics also play a role here, although they are more difficult to measure, the BMC Health Services Research authors said.

“The propensity to respond to surveys of patient satisfaction is likely multifactorial, and underlying patient characteristics or personality traits (other than general measures such as age or sex, for example) that lead to response (or nonresponse) remain largely speculative and difficult to assess,” they wrote.

However, understanding the demographic data, in addition to having a strong understanding of patient motivation and characteristics that indicate a survey responder, will help organizational efforts to garner high response rates.

How can organizations prevent non-response bias?

Ideally, non-response biases would not be an issue for healthcare organizations and patient satisfaction surveying. However, the data suggests most organizations do see at least a modest bias in their response rates, which in turn influences their practice decisions.

The first step to reducing the impacts of non-response bias is preventing it to begin with.

Offering surveys in the most common languages spoken at the organization; conducting more outreach to all patients, especially those less likely to respond to the survey; writing surveys in basic language to account for limited health literacy; and making surveys available in the clinic, via telephone, over the smartphone, and online may help close response gaps.

Organizations may decide which initiatives are best suited for their needs based on their non-respondent populations. However, organizations must be wary of where there is no immediate solution to non-response, the PLOS One article authors suggest.

“For cognitive limitations, mental illness or substance abuse, proxy respondents may be considered,” they wrote. “Proxy respondents usually provide reliable factual information and cause limited biases, but whether this holds also for subjective assessments such as satisfaction with health care requires further study. Health surveys do require a level of cognitive ability and motivation that may be out of reach for a substantial proportion of the target population. However strategies to improve participation can be implemented only if potential barriers are identified.”

Because of those limitations, it may be difficult for an organization to entirely rid their patient satisfaction surveys of non-response bias. Therefore, it will be important for organizations to be transparent about their satisfaction survey methodologies.

“Broadly, when any survey is widely used as the basis for an external quality indicator or comparison tool, the importance of utilizing sound and transparent methodologies in the development, the administration, and the interpretation of the survey becomes paramount,” the BMC Health Services Research article stated.

If a patient satisfaction survey will be informing a certain practice decision, it will be essential for survey administrators and other practice leaders to be up front with patients, other staff, and themselves about possible biases and the limitations they impose.