By Debra Semans, Senior Vice President, Polaris Marketing Research

One of the final steps in preparing data for analysis is to clean and code the responses that have been given to open-end questions. We all love to ask open-end questions, but coding the responses can be a frustrating and tedious task. Coding is basically the process of taking the open-end responses and categorizing them into groups that can then be used in analysis. The coding process is open to the judgment and interpretation of the coder, so it is something that must be done carefully. Here’s a process that will ensure you make the most of your open-end question responses.

A note about verbatims: If you need to enter the data for the open-end question (for example, from a mail survey), enter it exactly as written. Do not be tempted to skip this step, or to “code as you go”. You will want to have the raw verbatim recorded in case you need to come back to it later.

A note about technology: There is software that can be used to help you code open-end responses. We don’t use it, because it is not perfect, and still requires a lot of human labor to make sure the codes are appropriate. What we will do is use searches for key phrases or words, which can give you cues to potential category codes.

Start with the first question that needs coding. In my experience, respondents answer more completely earlier in the survey, and then tend to shorten their answers to open-ends that come later in the survey. So start with the ones that give you the most data to work with.

1. So the first step is to determine what kind of question you are dealing with. Are the responses truly from an open-end question or are they responses to “other, please specify” question? In the latter case, you are looking for answers that could be coded back into existing categories as well as additional categories that should be added. If they are true open-end questions with no existing responses, you’re starting from scratch. In either case, the process is generally the same, but you will probably end up with fewer new categories with an “other”-type question than with a true open-end. (A good rule of thumb is that no response category should account for less than 5% of individual responses.)
2. Next – read the responses. Reading through the responses will help you get a feel for potential response categories. Depending on how many respondents you have, how many answered the question, and how long the verbatim responses are, you may choose to read only a sample of the responses. Try to read at least 100. If you are only reading a sample, select randomly through the data set so that you don’t get a bias with early responses vs. later responses.
3. Based on your read, come up with some potential response categories. At this point, don’t worry about having “too many” categories. You will narrow them down later.
4. Go back to the open-end responses and try coding them. You may find it necessary to add more codes. Once you have them all coded, try to narrow the list of categories by combining those that seem to be similar. Try not to have more than 10 categories, with no individual category receiving less than 5% of responses.
5. Make sure all the responses are accounted for. Even if you end up with a few outliers in an “other” category, every response must go somewhere.
6. Create accurate and unambiguous codes, so that anyone reading the report will understand what kinds of comments are included in the category. It might also be useful to include a more detailed explanation with some sample verbatim responses in the report.
7. If a comment includes multiple topics, code it into multiple categories. Many times respondents put several comments into one answer. If only one of their answers is coded, the rest will be understated. (So, “Orange juice is delicious and nutritious” would be coded into both the “Delicious” and the “Nutritious” categories.)
8. Sometimes it is valuable to categorize the codes further into positive, neutral and negative groups, but this is not always the case. Consider if this adds information or not before proceeding.
9. Repeat as necessary.

Open-end question responses can be a very interesting source of information and many of Polaris’s clients like to receive verbatim reports (simply a word document with all of the verbatim responses laid out by question.) Additionally, when reporting on the coded open-end question, we will often pull out illustrative verbatim responses to draw additional attention to a key finding.  This “voice of the customer” can help bring your research report to life.

__________________________________

Debra Semans is the Senior Vice President for Polaris Marketing Research, with responsibility for Account Management, Marketing and Business Development. With more than 25 years of marketing experience, Debra brings rich and varied experience to her clients.