Request a quote!

Blog, News &

Case Studies

LearnBlog
Leveraging open-ends in market research questionnaires
By E2E Research | May 24, 2022

Most questionnaires have at least a couple open ends or verbatims in them. Places where research participants can share their thoughts or opinions in a text box instead of clicking on radio buttons or checkboxes.

 

When they get the final dataset, researchers who are quantitative at heart might quickly scan those verbatims to identify poor quality answers, delete those entire completed questionnaires, and then proceed directly to the quantitative analysis.

 

Is that the best use of verbatims? Why do we cast those questions aside so quickly? Well, there are lots of reasons but let’s start with two.

 

 .

Quantitative analysis is easy

Quantitative questions are much easier to count and summarize. You can say that a specific number of people clicked in a specific box to endorse a specific idea. You can run statistical tests, create charts and tables, and quickly see side-by-side whether there are differences of opinion among different people. It’s fast, easy, and can be done with minimal thinking – just follow the rules.

 .

 

Qualitative analysis is less precise

On the other hand, qualitative questions generate less precise answers. Words have nuance and diverse meanings depending on your background, experience, and understanding of the language at hand. It’s really hard to count and summarize opinions when 100 people say 100 different things. Coding and analyzing those answers takes a lot of time, particularly if multiple coders need to agree on an end result. Besides, some of the answers are garbage because some people don’t like typing or they have a hard time articulating their thoughts.

 

Is that a fair summary though?

 .

 

Imperfect precision plagues all question types

Of course not. Qualitative questions aren’t the only ones plagued with precision and validity issues. EVERY type of question, qual or quant, has it’s own set of issues affect precision, validity, and reliability.

 

Let’s consider the process a person takes to answer a very simple quantitative question, for example, “What kind of milk did you buy last week?”

  • They must interpret the words in the question: Could “you” be my partner or other household members? Does “last week” mean the last 7 days or this calendar week? Does it include baby formula? What about milk that was given to me, not bought? What if I bought milk 8 days ago?
  • They must interpret the words in the potential answers: Could “dairy milk” mean goat milk? Could “plant milk” mean nut milk? Does it include powdered milk? Does “other” include 8 days ago and never?
  • They must interpret the instructions: Does “Please specify” mean I need to share the brand name and the person who bought it and the date they bought it?
  • They must think about their own perspective related to that idea: Am I comfortable sharing this information? Do I have enough details to answer this question?
  • They must translate their answer into one that comes closest as possible to one of the existing options. And do this without accidentally choosing an answer in the wrong row or column.

 

For such a simple question about buying milk, there’s a lot of room for interpretation and misinterpretation. There’s a lot of room for imprecision. A lot of room for error. As much as we’d like to think there’s a cut and dry answer, that’s simply not the case for everyone.

 

Even the most simple quantitative questions come with nuance and imprecision.

 

 

Open ends are valuable work

Leveraging open-ends means that additional work falls into the hands of the researcher. WE must take the time to interpret answers, grasp the intended meaning, and properly categorize them. That’s a lot of work, especially when 1000 people have filled out an open-end.

 

At the same time, though, open-ends give participants the opportunity to share their thoughts in a way that truly reflects their whole self.

  • They can use swear words to demonstrate the intensity of their emotions. This is far better than seeing a 5 out of 5 on a Likert scale.
  • They can use slang to reveal their culture and social experiences. This is far better than inferring from their employment and education.
  • They can talk about issues that didn’t occur to you because your background, culture, and experience are completely different than theirs. This is true insight!

And, using open-ends shows participants we really do respect and value their opinions.

 

 

What’s Next?

Though incorporating open-ends requires additional time for analysis and coding, they have essential benefits for both the researcher and the participant. We can’t avoid using them because they aren’t “precise” or they aren’t “useful.” They are an essential part of the wise researcher’s toolkit. It’s time to cast aside our quantitative tendencies and truly embrace the open-end answers.

 .

.

 

Qualitative research podcasts You Might Like

 .

.

Qualitative research conferences You Might Like

 .

.

 

Learn more from our qualitative research case studies

 

Tags