Step 3: Determine the format of the response options

There are a wide variety of response options available to you. The response option you choose, however, should always be based on the objectives of the question and survey. Below you will find information about the most common response options. 

Yes/No options. These options are quick and easy to answer and allow you to generate simple comparisons. Very few answers, however, are ever truly dichotomous, containing no grey area in between. This type of question also provides you with very little depth of understanding. Consider the following:

  Have you used ITS services in the last month?
  __ YES       __ NO
  A student may have used the service in the last month, but a simple yes/no response will not provide an understanding of the extent of that use. For this reason, yes/no options should be only used on rare occasions. One such occasion is for contingency questions. This type of question first prompts the respondent with a yes/no question to determine if subsequent questions apply. 

Multiple choice options. These options provide a fixed set of answers to choose from and can be designed to allow the respondent to select only one or multiple response options. They are often quick and easy to answer and can be good for collecting factual information. One disadvantage, however, is that by providing fixed answers, you may miss an important factor that you failed to otherwise consider. If you decide to use multiple choice options, keep the following in mind:

  • Response options should be mutually exclusive. All of the options you include should be clearly distinct from one another, with no overlap. If the options are not distinct, respondents will not know which option to select. Consider the following:
    Poor Example: How many hours did you study for this class in the last week?
      __ 0 hours       __ 1-3 hours       __ 3-6 hours       __ More than 6 hours
    Which option would a student select if they studied 3 hours? Recategorize the options to be mutually exclusive:
    Better Example: __ Less than an hour       __ 1-3 hours       __ 4-6 hours       __ More than 6 hours
  • Capture all possible responses. Strive to include all responses while using the smallest number of categories possible. The “other” category is useful to capture options that you may have overlooked. Include this at the end of the list of response options. Consider the following: 
    Poor Example: What is the highest level of education your mother completed? 
      __ Graduated from high school 
    __ Completed bachelor’s degree (B.A., B.S., etc.)
    __ Completed master’s degree (M.A., M.S., etc.)
    __ Completed doctorate degree (M.D., J.D., Ph.D., etc.)
    This question provides no option for those that did not complete high school, attended college but did not complete a degree, or those that completed an associate’s degree. These options are common enough that they should be included in the response options itself and not simply through the use of an “other” category:
    Better Example: __ Did not complete high school
    __ Graduated from high school
    __ Attended college but did not complete degree
    __ Completed associate’s degree (A.A., A.S., etc.)
    __ Completed bachelor’s degree (B.A., B.S., etc.)
    __ Completed master’s degree (M.A., M.S., etc.)
    __ Completed doctorate degree (M.D., J.D., Ph.D., etc.)
  • Arrange the options logically. Arranging the options logically will reduce confusion and the chance that respondents may overlook an option. For instance, when using ordered categories, place the options in an increasing order. Consider the following:
    Poor Example: What is your current class year?
      __ Graduate 
    __ Freshman    
    __ Senior      
    __ Junior     
    __ Sophomore
    It can take a respondent a longer amount of time to navigate through this question as the response options are not in a logical sequence. Place the options in an increasing order:
    Better Example: __ Freshman     
    __ Sophomore 
    __ Junior     
    __ Senior      
    __ Graduate 

Likert scales. These response options are used when measuring opinion-based questions. Respondents are asked to rate their preferences, attitudes, or subjective feelings on a scale. One benefit of using this type of scale is that many respondents are familiar with the format and will find them easy to complete. They can also provide you with a good deal of information. If you decide to use Likert scale response options, keep the following in mind:

  • Use a 5-6 point scale. The typical Likert scale includes 5-6 points, with the 6th point reserved for the “don’t know” response. A smaller scale runs the risk of not capturing the respondent’s choice, while a larger point scale can lose meaning as the difference between one point and another is minimized. Consider the following: 
    Poor Example: On a scale from 1-100, 1 being “strongly disagree” and 100 being “strongly agree,” rate your level of agreement or disagreement.
    This scale is so large that the respondent will not consider the difference between the points to be significant. Here is a good example of a 6 point Likert scale:
    Better Example: 1
    Strongly 
    Disagree
    2
    Disagree
    3
    Neither Agree 
    nor Disagree
    4
    Agree
    5
    Strongly
    Agree
    6
    Don't 
    Know
  • Describe each point. Provide a description for each point on the scale. Unlabeled numbers on a scale can have different meanings to respondents. Labeling the scale eliminates the possibility of misunderstanding, as the choices are explicitly stated. 
  • Consider a Neutral Category. Neutral types of responses are placed in the middle of the scale and indicate familiarity with the topic but no opinion one way or the other. Keep in mind that by not including this option, you will force respondents to have an opinion on the topic. Note that the “don’t know” option is not equivalent to neutral options. “Don’t know” indicates that the respondent is unfamiliar with the topic or refuses to answer. 
  • Arrange the options logically. As with multiple choice options, the options for Likert scales should be presented in a logical order. After you select the ordering of response options, be consistent in your ordering for all similar questions in the survey. 

Open-ended responses. Open-ended responses allow the respondent to provide their own free-form answer. For example: 

  What aspect of this course was most beneficial to your learning?
  __________________________________________________
  These types of responses can provide you with a rich source of information. They are most useful when there are many possible responses to a question or when you wish to probe deeper into an issue. Despite these advantages, open-ended responses are time-consuming. It will take longer for the respondent to answer these types of questions and it will take you longer to read and analyze the responses. For this reason, open-ended questions are typically used sparingly in surveys.

 

Alternative responses. Several alternative response options are commonly used in surveys to allow the respondent to essentially “opt-out” of answering or provide their own answer to a question. There is no one right answer when deciding to include these responses. Use your best judgment, based on your objectives and knowledge of the survey population. Here are some issues to consider when deciding to use some of the most common alternative responses:

  • “Don’t Know” and “Not Applicable.” These alternatives allow a respondent to opt-out of answering a question if they are not familiar with the topic or the topic does not apply to them. Consider the following when deciding to use these alternatives:
    • If a respondent is not familiar with the topic or if the topic does not apply to them, they should not be forced to provide a response. Including these alternatives will allow you to capture their response accurately.
    • On the other hand, you can never be certain why a respondent selected these alternatives. It could be that the respondent was not familiar with the topic of interest, did not understand the question, or simply selected this to quickly finish the survey. The only thing that you can be certain of is that the respondent decided to opt-out of the question.
  • “Decline to State.” Providing this alternative allows respondents to formally refuse a question. Consider the following when deciding to use it:
    • Certain question topics can be sensitive to some individuals. For instance, not all respondents will be comfortable providing their income on a survey. In these cases, using the “decline to state” option will help to reduce the chance that a respondent will become upset by the question and abandon the survey altogether. 
    • As with the “don’t know” and “not applicable” alternatives you can never be certain why a respondent selected this option, only that they decided to opt-out of the question.
  • “Other.” This alternative response allows the respondent to provide their own answer to a question if their answer does not fall under one of the provided response options. It can be extremely beneficial when you are uncertain as to whether you have captured all possible responses. Similar to open-ended responses, this alternative can be time consuming. Consider limiting its use to those instances where you believe important information would be missed otherwise. Here’s an example of one such instance: 
      What types of professional communications organizations do you belong to? (Mark all that apply)
    __ International Communication Association
    __ National Communication Association 
    __ Public Relations Society of America
    __ Society of Professional Journalists
    __ Other(s), please specify _____________________________________
      In this example, a Communication Studies program would like to know how their alumni stay connected to the field after graduation. They included a list of common associations, but provided an “other” response for less common associations. This enables them to provide respondents with common options, while not overloading them with a long list of choices.

 

Reference

 

Suskie, L. A. (1996.) Questionnaire survey research: What works. (2nd ed.). Tallahassee, FL: The Association for Institutional Research.

Return to Survey Design homepage                                                                                Next: Step 4: Format the survey