Researchers want people to feel comfortable when they’re answering questions so they will share complete and honest answers about their public and private lives. This can be difficult because you first need to create a research environment that shows them they are trusted and respected.

If this resonates for you, then please join our Chief Research Officer, Annie Pettit, for a 3-day (2 hours per day) questionnaire design workshop hosted by Esomar on February 21, 22, 23, at 13:00 UTC.

In this highly interactive masterclass, you will learn about the psychology of answering questions and how to apply that knowledge in a practical way to questionnaire design. You will learn to create questions that make people feel valued and respected, that accommodate normal human behaviours no matter how strange those behaviours may seem to you, and that make people look forward to participating in the next research project.

Registration is now open on the Esomar website!

What will you learn?

After completing this training, you should be able to:

• Understand how human psychology interacts with the questionnaire experience and how to write questions that accommodate normal behaviours
• Write questions that are kind and respectful towards people who are marginalised, and people who are embarrassed to share personal aspects of their lives
• Write questionnaires that people want to answer this time and the next
• Write questions that are fun and playful

By the end of the workshop, you will have built a set of resources that can be leveraged in a range of future questionnaires.

Programme at a glance

Session 1: Understanding human nature

• How does human psychology impact how people interpret and respond to questionnaires
• What are some basic rules for creating respectful questionnaires
• What mindset do we need to take when building questionnaires

Session 2: Making tough topics more comfortable

• Creating compassionate screener questions
• Writing respectful data quality questions
• Asking about embarrassing or private issues
• Eliciting truth when it comes to unethical and illegal behaviours

Session 3: Creating a playful experience for everyone

• How to write playful questions that excite and make people think
• Incorporating play into serious topics

Please click here to register on the Esomar website.

About Annie Pettit

Annie Pettit is Chief Research Officer, North America, at E2E Research, an ISO 27001 certified, ESOMAR corporate member company that offers market research, data analytics, and business intelligence solutions to help research leaders understand their buyers, brands, and businesses. Annie is a research methodologist who specializes in research design and analysis, data quality, and innovative methods. She holds a PhD in experimental psychology from York University in Canada, is a Certified Analytics and Insights Professional (CAIP), and is a Fellow of the Canadian Research Insights Council (CRIC). Annie is also Chair of the Canadian ISO Standards Committee (ISO 20252), and the author of “People Aren’t Robots: A practical guide to the psychology and technique of questionnaire design.”

Hi and welcome!

On Tuesday, September 20th, Rupa Raje and Annie Pettit ran a session at Esomar congress on designing more playful questionnaires. In preparation for that presentation, researchers were invited to complete a very short questionnaire incorporating a range of innovative question designs.

That questionnaire is now closed but you can read some of the results in our article on Esomar’s Research World. Enjoy!

Most questionnaires have at least a couple open ends or verbatims in them. Places where research participants can share their thoughts or opinions in a text box instead of clicking on radio buttons or checkboxes.

When they get the final dataset, researchers who are quantitative at heart might quickly scan those verbatims to identify poor quality answers, delete those entire completed questionnaires, and then proceed directly to the quantitative analysis.

Is that the best use of verbatims? Why do we cast those questions aside so quickly? Well, there are lots of reasons but let’s start with two.

 .

Quantitative analysis is easy

Quantitative questions are much easier to count and summarize. You can say that a specific number of people clicked in a specific box to endorse a specific idea. You can run statistical tests, create charts and tables, and quickly see side-by-side whether there are differences of opinion among different people. It’s fast, easy, and can be done with minimal thinking – just follow the rules.

 .

Qualitative analysis is less precise

On the other hand, qualitative questions generate less precise answers. Words have nuance and diverse meanings depending on your background, experience, and understanding of the language at hand. It’s really hard to count and summarize opinions when 100 people say 100 different things. Coding and analyzing those answers takes a lot of time, particularly if multiple coders need to agree on an end result. Besides, some of the answers are garbage because some people don’t like typing or they have a hard time articulating their thoughts.

Is that a fair summary though?

 .

Imperfect precision plagues all question types

Of course not. Qualitative questions aren’t the only ones plagued with precision and validity issues. EVERY type of question, qual or quant, has it’s own set of issues affect precision, validity, and reliability.

Let’s consider the process a person takes to answer a very simple quantitative question, for example, “What kind of milk did you buy last week?”

• They must interpret the words in the question: Could “you” be my partner or other household members? Does “last week” mean the last 7 days or this calendar week? Does it include baby formula? What about milk that was given to me, not bought? What if I bought milk 8 days ago?
• They must interpret the words in the potential answers: Could “dairy milk” mean goat milk? Could “plant milk” mean nut milk? Does it include powdered milk? Does “other” include 8 days ago and never?
• They must interpret the instructions: Does “Please specify” mean I need to share the brand name and the person who bought it and the date they bought it?
• They must think about their own perspective related to that idea: Am I comfortable sharing this information? Do I have enough details to answer this question?
• They must translate their answer into one that comes closest as possible to one of the existing options. And do this without accidentally choosing an answer in the wrong row or column.

 

For such a simple question about buying milk, there’s a lot of room for interpretation and misinterpretation. There’s a lot of room for imprecision. A lot of room for error. As much as we’d like to think there’s a cut and dry answer, that’s simply not the case for everyone.

Even the most simple quantitative questions come with nuance and imprecision.

Open ends are valuable work

Leveraging open-ends means that additional work falls into the hands of the researcher. WE must take the time to interpret answers, grasp the intended meaning, and properly categorize them. That’s a lot of work, especially when 1000 people have filled out an open-end.

At the same time, though, open-ends give participants the opportunity to share their thoughts in a way that truly reflects their whole self.

• They can use swear words to demonstrate the intensity of their emotions. This is far better than seeing a 5 out of 5 on a Likert scale.
• They can use slang to reveal their culture and social experiences. This is far better than inferring from their employment and education.
• They can talk about issues that didn’t occur to you because your background, culture, and experience are completely different than theirs. This is true insight!

And, using open-ends shows participants we really do respect and value their opinions.

What’s Next?

Though incorporating open-ends requires additional time for analysis and coding, they have essential benefits for both the researcher and the participant. We can’t avoid using them because they aren’t “precise” or they aren’t “useful.” They are an essential part of the wise researcher’s toolkit. It’s time to cast aside our quantitative tendencies and truly embrace the open-end answers.

 .

.

Qualitative research podcasts You Might Like

• Between the Data – NVivo Podcast Series
• The New Mainstream Podcast
• Ethnography Atelier Podcast
• Qualitative Conversations
• The Qualitative Escape
• Research with Dr. Anil
• An Eye For Detail

 .

.

Qualitative research conferences You Might Like

• QUAL360 North America, March
• International Congress of Qualitative Inquiry, May
• QRCA Annual Conference, May
• The Qualitatives, June

 .

.

Learn more from our qualitative research case studies

• Face-to-Face On-Site Medical Leader Interviews – A B2B interview case study
• Sentiment and Content Analysis of Qualitative Consumer Comments – A food survey + text analysis case study
• Sentiment and Content Analysis of Qualitative Hiring Data | A qualitative research case study

Hosted by Jenn Vogel at Voxpopme, Reel Talk is a podcast designed to help researchers and marketers gain valuable information to help understand their customers better. Jenn and her guests discuss customer insights and how to use data to make better customer-centric decisions. You can catch all of the podcasts here.

In this episode from October 4, 2021, Jenn chats with Annie Pettit, our CEO, about techniques for designing better marketing research questionnaires.