im
classic-banner

Battling Survey Bias

Which is more difficult to control -- the weather or bias in a research survey? That question would be a real stumper to many in the research field.�Some types of survey bias are easier to know and manage than others, but as a whole, the topic is a difficult one that affects all survey modes.

If we're talking about question/response bias, then there are plenty of techniques that can be used to mitigate this bias. Some frequently used methods are:

  1. Write questions that are clear, precise, and relatively short

  2. Do not use �loaded� or �leading� questions

  3. Avoid double-barreled questions

  4. Avoid double negatives

  5. Use both mutually exclusive and exhaustive response categories for closed-ended questions

  6. Reverse the wording in some of the questions to help prevent response set bias

  7. Rotate the responses

These are very commonly and successfully used to avoid response bias, but what about nonresponse bias?�Polaris Marketing Research defines nonresponse bias as "Error that results from a systematic difference between those who do and do not respond to the measurement instrument."

There are many schools of thought and opinions on the degree of impact of nonresponse bias on survey results.�On the University of Florida's website, Glenn Israel reports that, "As nonresponse increases, the potential for a biased sample increases.�This means that the obtained responses of a probability sample may no longer be representative of the larger population."�Nonresponse impacts surveys in overall bias, item bias and in making the data collection period longer (which can introduce more bias).

Short of throwing the data away, here are some options to understanding and managing nonresponse bias when the data has been collected.

  1. Generalize to the respondents only.

  2. Assume there is no response bias and generalize to the population.

  3. Use a different methodology to re-contact nonrespondents to get their information.

  4. Compare data in hand on respondents and nonrespondents (e.g., demographics, customer information).

  5. Compare characteristics of early respondents with late respondents (who are more likely to be similar to nonrespondents).

  6. Increase mailings or contact efforts.

In most cases, researchers don�t have much information about nonrespondents, so the best way to address nonresponse bias is to encourage response in the design and management of the survey. Some ways of doing this are to:

  1. Use cash incentives.

  2. Use multiple follow-up contacts.

  3. Use short, easily understood surveys when possible.

Overall, the response and nonresponse biases are aspects that must be addressed in every survey in order for the results to be meaningful.�Question/response bias can be more easily controlled, while nonresponse bias requires forethought, great care and intuition, and even then can be a source of uncertainty.

All Content � Copyright 2004-2011. Polaris Marketing Research, Inc. All Rights Reserved.

Send inquiries to newsletter@polarismr.com or call 1-888-816-8700.

ec