im
classic-banner

Pros and Cons of Different Research Methodologies

So you�ve decided to conduct market research, but now you have to settle on a survey methodology. You have heard that web surveys are the cheapest option, but surely there must be a tradeoff for that low cost advantage. And what about phone surveys � how can you justify spending so much more when there are cheaper options available? Your choice will come down to several different methodologies that fall into two major groups � self-completion surveys and interviewer-assisted surveys.

Phone Surveys?

Phone surveys are the most common version of interviewer-administered methodologies. They are usually assumed to provide more reliable results and response rates than self-completion methods, as there is an interviewer involved in the process that can help to encourage a respondent to participate in a survey and keep them focused on the correct subject matter during the process. Questions on a self-completion survey often leave room for interpretation of their meaning by a respondent � having an interviewer present during phone surveys helps to counteract any of these misunderstandings. Another perk of having an interviewer is that they can guide respondents through an advanced survey structure where some questions may be skipped based on previous responses, which can be more difficult in some self-completion methodologies.

A particularly useful feature of phone surveys is the ability for the researcher or client to monitor, or �listen in,� to surveys as they are being taken. This provides insight into the survey experience from a respondent�s perspective, and may help to improve the structure of an ongoing survey over time.

Despite being the more reliable data collection methodology, phone surveys have several drawbacks. The most obvious of these is the increased cost. Interviewer-assisted survey providers have significantly more overhead than self-administered vendors because of the costs of running a phone center. These expenses are passed on to clients, creating a much higher per-interview cost than self-administered methodologies.

Phone surveys also tend to make respondents soften any harsh responses they may want to give about a product or service. Many will assume that an independent interviewer is part of the company they are being asked about, and will avoid giving any extremely low ratings.

Other Interviewer-Assisted Methodologies

Other interviewer-administered data collection methodologies include intercept or exit surveys, door-to-door interviewing, and central location tests. All of these suffer from limitations in being able to collect data from a representative sample of respondents, and also from the cost of interviewing. Additionally, concerns about personal security and privacy make these methodologies less attractive.

Web Surveys

Web-based surveys are the most popular method of self completion surveys. They maintain the advanced survey structure advantages of an interviewer-administered survey while reducing the cost-per-respondent by lowering data collection (variable) costs. A survey conducted by General Mills suggests that online methodologies can reduce research costs by an average of 50 percent. However, this reduction may not be present in every project and is very dependent on sample size, the length of the questionnaire, and respondent type.

Internet technology provides researchers with tools that are not readily available in other methodologies. Multimedia presentations and other graphic elements can be incorporated into web surveys to test a respondent�s reaction to visual stimuli. Additionally, Internet survey technology allows the researcher to control when and how the respondent is exposed to the stimuli. This can prove especially useful in concept testing. Web surveys also provide researchers with more advanced question techniques that would normally be too tedious for an interviewer-based survey, such as maximum difference and discrete choice analyses.

The lack of a live interviewer in web surveys helps to moderate the respondent bias often found in interviewer-assisted surveys, as respondents are less likely to soften their responses to avoid being rude. Self-completion also significantly decreases the average length of a survey. General Mills also found that the time it takes to complete a typical survey is reduced by as much as two-thirds when a web methodology is adopted. Additionally, the ability to stop the interview and resume at the stopping point is very convenient for time-pressed respondents.

While web surveys have proven to be cheaper than their interviewer-based counterparts, there are still several significant drawbacks to consider. As with any self-completion methodology, respondents are less likely to participate if they must initiate the survey. Web response rates of 15 percent are considered to be good, while rates below 10 percent are closer to the norm. This is typically lower than rates found in interview-based surveys, where response rates of 20 percent and higher are the norm. Also, the amount of Internet penetration in the population must be considered. While the number of non-Internet users is certainly shrinking, there is a chance that the population being surveyed is not Internet savvy, which could lead to even lower response rates.

While helping to reduce bias and completion times, the lack of a live interviewer to guide the process also presents several problems. First, respondents are more likely to �drop off� a survey, as there is no feeling of guilt that could be present when terminating an interviewer-administered survey. Second, web surveys, like all self-completion methodologies, may represent only extremes of opinion. Those who have very strong positive or negative feelings are more likely to put forth the effort to initiate a self-completion survey than more neutral respondents.

Other Self-Completion Methodologies

Direct mail is a type of self-completion survey that has seen a decline in popularity with the increased use of web surveys. Respondents are mailed a paper survey with a return envelope enclosed, and then asked to return the survey in a given time frame. Like web surveys, direct mail is a cheaper option than phone surveys because of the lack of an interviewer and call center. However, respondents are even less likely to complete the survey than they would be to complete a web survey because of the added step of mailing upon completion. This, combined with increasing printing and postage costs and the larger number of surveys that must be mailed to counter the very low response rate (5 percent to 7 percent, typically) make mail surveys much less of a bargain than they were in the past.

IVR (Interactive Voice Response) allows respondents to call a toll-free number and complete a survey using a touch-tone phone. Like web surveys, IVR keeps the advanced survey structure of phone surveys while reducing cost. However, it faces the same major flaw as all other self-completion methodologies � respondents must initiate contact, which greatly reduces response rates (usually about 5 percent or less).

Making the Right Choice

The most important thing to understand in choosing a research methodology is that there is no single correct answer. The decision must be addressed on a per project basis, and should not be based solely on budget. One must weigh the pros and cons of all available methodologies and select the option that has the best chance of reaching the desired population and achieving the research objectives.

All Content � Copyright 2004-2011. Polaris Marketing Research, Inc. All Rights Reserved.

Send inquiries to newsletter@polarismr.com or call 1-888-816- 8700.