Avoid Bias in Survey Design

HARDWICK RESEARCH “TALKS TRASH”

After conducting surveys for 20+ years, Hardwick Research knows what it takes to develop a questionnaire that removes the possibility of bias and is organized to maximize the quality of responses. To help you avoid bias in survey design and not have your research be “trash,” we compiled a list of the 8 most commonly seen mistakes.

1. Limited multiple choice answers:  It’s always good to have an “Other (specify)” option to allow respondents to answer the question with the response that makes the most sense to them.  If the list provided does not contain their desired response, the respondent will choose something (even if it’s the wrong answer) just to move on.
2. Asking leading questions: Leading questions are often asked by a researcher who isn’t thinking through their survey. They know they want the respondent to answer a particular way, so they write the question that way. Asking a leading question will provide a false response. To avoid this, think critically about your question and make sure it is not biased in any way. It’s always a good idea to have someone else review your questionnaire before launching the survey.
3. Too many open-ended questions: Open-ends are time consuming for respondents to complete and typically do not include the level of detail you are hoping for. If you overload the survey with open-ended questions respondent fatigue will set in.
4. Including unimportant questions: Be sure that each question being asked is necessary in order to reach your objectives. For each question ask yourself what you will learn from that question, how you will use the results, and if it is critical to know this information. If it’s not critical, then remove the question. Keep in mind that the longer the questionnaire, the more expensive the research. Plus longer questionnaires lead to respondent fatigue and poor quality results.
5. Asking two questions at once: Asking a respondent to rate the “overall quality and reliability” of a product asks them to consider two aspects in one rating. These are actually two different issues. For example if this is rating an electrical utility, the quality of the electricity might be great (when it’s on it works fine), but the service is not reliable (too many power outages).
6. Beginning with demographic questions: Obviously if you have to screen for a particular type of person, you will need to include some demographic questions up front, but the remaining demographic questions should always be at the end. You want to engage the respondent. If you are asking them to participate in a research survey on pet food, don’t start by asking them their education or income.
7. Ambiguous questions: Be very clear in what specific information you are looking for. If you want someone’s zip code, don’t ask them where they live.
8. Assuming knowledge: You know your company inside and out, but your customers and prospects do not. Don’t assume the participant knows a lot about your company or the industry. Stay away from industry specific words or abbreviations.

To avoid bias in survey design and “trash” for results, have an experienced researcher review your questionnaire before launching it. Pretest your questionnaire with a small group of your target audience first to make sure the survey is clearly written. At a minimum, have someone in your office who didn’t write the survey complete it and provide feedback.

Feel free to contact Hardwick Research. We are experts at removing bias from your research surveys. We are always happy to write or review a questionnaire for you, even if you don’t need assistance with the rest of your project.