Market research is all about disrupting the status quo.

We aim to discover what the average person likes or dislikes about a brand and then improve those things to increase our chances of success. We try to change perceptions about brands people think they don’t like so that they’ll switch to our brand. We work really hard to change other people’s attitudes and behaviors.

But what are we changing about ourselves?

Humans use routine and cognitive biases to simplify and speed up their decision-making processes, and marketing research is overflowing with those routines.

Why do we use the same 30-minute tracker or U&A questionnaire every year?

Sure, we want to create norms and benchmarks so we can see what has changed and where we’ve improved. But how could it possibly be that we were so incredibly smart that THIS 30-minute tracker is precisely, exactly what we need forever and always?

• Perhaps today is a great day to change it into three 10-minute questionnaires so that people can remain fully engaged through-out and not worry about finishing in time to pick-up their kids.
• Perhaps today is a great day to remove the 10 minutes of questions that we’ve never acted on.
• Perhaps today is the day we throw the whole thing in the recycle folder and brainstorm a completely new, far more engaging and actionable way to ask about our brand.

Why do we always default to questionnaires/groups/social listening?

Sure, we’ve written so many questionnaires that we’re experts at it. We could write a 30-minute questionnaire in 20 minutes. So how come someexpert researchers have never written one?

• If you’ve never personally participated in a focus group, maybe now is the perfect time to try one so you can truly dig deep into the psyche of your buyers and users.
• If you don’t trust biometrics, why not try one study just to prove yourself right – or wrong.
• Why not take a class in interviewing or moderating so you’ll be just as comfortable defaulting to qualitative research when that method better suits the research objective?
• Why not build a better tool box so you’ll be able to gather the precise insights you need rather than the insights that happen to be available at that moment?

Why do we always use the same sample sizes?

How is it statistically possible that n=300 is the best sample size to choose a nice color for a package and also the right sample size to choose a drug that will save 5 out 100,000 lives rather than just 4?

• You may not like statistics but maybe the true statement is you don’t like statistics yet. Finding the right instructor can make all the difference.
• Have you learned in detail what effect sizes are and how they affect sample sizes? Maybe 300 is correct, but maybe you actually only need 200.
• Have you tried running the exact same study twice but with sample sizes of only 30? Seeing just how different those results are is a good reminder that random chance is real, and reliability and validity are important for generating actionable recommendations.
• Have you tried generalizing research findings from a community survey of 30 Black people to 300 million white people? Over-sampling under-represented people is not a luxury – it’s a necessity.

There are so many good reasons to create time-saving templates and cognitive rules, but also so many reasons to ignore them. You need to ensure you’re making the choice for yourself rather than defaulting to a template for reasons of speed. That’s how we end up making great decisions.

Podcasts You Might Like

• Change by Attraction with Esther Derby
• Change on the Run with Phill Buckley
• Change Management Review
• Conversations of Agile Change with Dr Jen Frahm and Lena Ross
• Two Minutes of Talking Change with Jennifer Campbell

When you’re immersed in data and numbers every day, all day long, it’s easy to forget that numbers can be intimidating. However, built with care and purpose, real-time dashboards are a great way to help non-technical people feel more comfortable with numbers and encourage them to dig into real-time data without feeling overwhelmed.

Regardless of how comfortable people are with numbers, everyone needs to understand and analyze their KPIs and critical data points to make more informed decisions that will result in business growth. As with any tool, there are good reasons to choose one data presentation tool over another.

With that in mind, let’s first consider under what circumstances dashboards are preferable and second, how to set up an actionable dashboard that people will want to use.

Optimal Use Cases for Digital Dashboards

Huge Sample Sizes

No one likes a long, PPT report. But when sample sizes are huge, forcing a potentially massive set of results into a short static report can minimize the potential of the data you so carefully collected. Think about it in terms of a global report covering a brand 15 different countries. It doesn’t make sense to write 15 reports that are each 20 pages. However, it does make sense to capture high-level global insights in one report and then provide dashboard access to the nuanced results within each country.

• Trackers that accumulate thousands of records on a daily, weekly, monthly, or quarterly basis
• Global point-in-time studies covering many SKUs, languages, and countries
• Transactional/purchase datasets covering hundreds of SKUs, hundreds of retailers, and millions of individual, consumer purchases

Time-Dependent Reporting

Whether it’s tracker data from the last 6 months or historical, business records from the last 6 years, dashboards can help you consolidate terabytes of data into meaningful chunks. Discover insights that have been hidden in the data because the data wasn’t previously reviewed with a certain question in mind or because year-over-year data wasn’t previously available.

• Monitor brand health and campaign effectiveness year-over-year
• Monitor seasonal employee satisfaction and engagement.
• Review the past, monitor the present, and predict the future

Access Real-Time Insights

When you’ve waited 4 weeks since the start of a project, 2 weeks since it went in field, and you still have to wait 2 more weeks until tabulations and a draft report is ready, you know the power of accessing real-time data. Dashboards can be the answer to quick insights, particularly when a problem appears seemingly out of nowhere!

• Identify problematic business practices and roadblocks from transactional or logistics data in real time
• Catch consumer-reported problems in social media data or tracker data before they become full-blown crises

Mine for Insights

It’s impossible to anticipate every possible, meaningful analysis prior to writing a report. With a user-guided dashboard, you can check hunches, test wild scenarios, and discover insights that were secondary (or tertiary) to the original research questions or that weren’t obvious at the time of writing. And, these analyzes can be done even by those who don’t have access to or knowledge or SPSS, SAS, or the original data tables.

• Dig into to data beyond the original research objectives
• Uncover serendipitous insights that would never otherwise be discovered

Reach Multiple Audiences

Most written reports are tailored for a single audience. But we know that research data is invaluable to many groups of people. With an interactive dashboard, each user can focus on the level of detail that will help them make the best decisions in their role, and all of them can be using the same raw data source for a consistent message.

• Sales/Marketing Team: Dashboards can help you understand the performance of individual salespeople, track the pipeline and conversion, understand marketing campaigns. All of this will help them understand how they are performing and where they need to direct their efforts.
• Brand Managers: Brand managers rely on analytical dashboards to track campaigns, product development, customer satisfaction, and more. Dashboards help them track key metrics and spot and resolve issues before they become much bigger problems.
• Operations Managers: Operations managers rely on operational dashboards to track purchase behaviors, discover logistical roadblocks, and improve processes.
• Decision-Makers: CEOs need a strategic dashboard with KPIs across all departments to track company goals, visualize new trends, and inform future strategies – all in one place.

Fuse Data from Multiple Sources

If you’ve ever struggled through 3 reports written by 3 different people in 3 different formats and tried to consolidate trends and themes, you know how valuable inputting all that data into one dashboard can be. Save time and confusion by incorporating website analytics, transactional data, survey tracker data, and customer support data into one place to reveal holistic, company-wide insights.

• Merge transactional and survey data for a holistic picture of customer
• Merge employee engagement data and sales data for a holistic picture of the business

Detailed Building Blocks for a User-Friendly, User-Guided Dashboard

After you’ve decided that an interactive, user-guided data dashboard is the right reporting tool for your research, then you need to actually build that dashboard. Here are a few key tips to keep in mind during the development process.

• Choose play: People want to play, even adults! Dashboards don’t have to be boring just because they’re designed for business professionals. Incorporate pleasing designs and interactive filters that encourage play and discovery. A playful dashboard is a used dashboard!
• Choose clean data: Don’t assume that all data is good data, and that all data can be immediately dropped into a dashboard. Check all of the data for errors, both manual and systematic, before loading it into the dashboard and letting users work with it. Make sure it’s clean, complete, and compliant. Don’t let the data lie to users.
• Choose the most important data: Yes, you can have a dashboard with 100 filters and 50 pages. But will they all be used? As the dashboard creator, you know which variables are of key importance. Focus on those so that users don’t get distracted by incidental data.
• Choose actionable data: If you know that you can never act on a certain issue, then it’s a waste of time, space, and users’ cognitive power to include it in a dashboard. Focus on data that people can and will act on to improve the business.
• Choose the right charts not the pretty charts: The purpose of a dashboard is not to include one of every type of chart. The purpose is to choose charts that are best suited to the data being shared. If that means one page has 5 line charts and no bar charts or pie charts, then so be it. Clarity is key.
• Choose accessibility: Sometimes, accessibility is easy. Make sure to use large fonts, comprehensive labels, indicators that can be differentiated in both black/white and color, generous spacing, and large clickable areas. Consider whether your audience has unique accessibility needs due to a disability. Even better, consult with an accessibility expert.

Types of Market and Consumer Insight Dashboards

No matter what kind of dashboard you need, you will be available to find a solution. If you can focus on your audience and your goal, you’ll be able to properly distinguish between three major categories of dashboards.

• Quick: When budgets are tight, timelines are short, and you still need a user-driven tool to investigate data and discover insights, try a quick and cheap dashboard. They may not have the swoopy transitions or endless bonus features but you can still get the basic functionality you truly need to analyze a few waves of tracker data or a multi-country study. Our Raven dashboards are one example of a quick and competitively priced dashboard.
• Comprehensive: For most people, the middle option works best. With tools like PowerBI (cost-effective for Microsoft users) and Tableau (super-speed with massive datasets), most medium to large datasets can be nicely transformed into easy to use, attractive dashboards.
• Custom: The sky is the limit! With tools like .NET and Python, you can have the dashboard of your dreams. Filter real-time transactional, survey, and logistics data into one dashboard. Forecast future sales given consumer opinion scores and live purchase data. Plan more timely deliveries of the SKUs they actually want.

What’s Next?

Once you’ve decided to use a dashboard, the sky is the limit. Focus on your needs not your wants, and you’ll end up with a dashboard that will help you gather insights into your buyers, brands, and business, and create a successful future.

Are you ready to build a quick, comprehensive, or custom dashboard that helps you communicate more effectively with a wide range of key stakeholders? E2E’s Raven dashboards are competitively priced even for small projects.  Email your project specifications to our research experts using Projects at E2Eresearch dot com.

Learn more from our case studies

• Automating Distribution of BFSI Performance Metrics – An automated reporting case study
• Creating a Financial Portfolio Health Review Report – A credit card business analytics case study
• Building a Strategic Business Plan Grounded in Industry Data – A pharmaceutical data analytics case study

Learn more from our other blog posts

• New! Raven Dashboards for Inquisitive Minds
• 8 Engaging Question Types to Improve Participants’ Survey Taking Experience
• Trackers Suck. Here’s how to fix them.
• 8 Reasons to Invest in a Hybrid DIY Market Research Team
• (Not-So) Secret Strategies for Great Market and Consumer Research

In the market and consumer research space, there is good data and bad data.

Good data comes from research participants who try to do a good job sharing their thoughts, feelings, opinions, and behaviors. They might forget or exaggerate a few things, as all people do every day, but they’re coming from a good place and want to be helpful. In general, most people participating in market research fall into this category. They’re regular, every day people behaving in regular every day ways.

Bad data comes from several places.

First, sometimes it comes from people who are just having a tough day – the kids need extra attention, the car broke down, their credit card was compromised. Some days, people aren’t in a good frame of mind and it shows in their data. That’s okay. We understand.

Second, rarely, bad data comes from mal-intentioned people. Those who will say or do anything to receive the promised incentive.

Third, very often, it comes from researchers. Questionnaires, sample designs, research designs, and data analyses are never perfect. Researchers are people too! We regularly make mistakes with question logic, question wording, sample targeting, scripting and more but we always try to learn for the next time.

In order to prevent bad data from affecting the validity and reliability of our research conclusions and recommendations, we need to employ a number of strategies to find as many kinds of bad quality data as possible. Buckle up because there are lots!

Data Validation

What is data validation?

Data validation is the process of checking scripting and incoming data to ensure the data will look how you expect it to look. It can be done with automated systems or manually, and ideally using both methods.

What types of bad data does data validation catch?

Data validation catches errors in questionnaire logic. Sometimes those errors are simply scripting errors that direct participants through the wrong sequence of questions. Other times, it’s unanticipated consequences of question logic that means some questions are accidentally not offered to participants. These problems can lead to wrong incidence rates and worse!

How do data validation tools help market researchers?

Automated systems based on a soft-launch of the survey speed up the identification of survey question logic that leads to wrong ends or dead ends. Manual systems help identify unanticipated consequences of people behaving like real, irrational, and fallible people.

Automated tools can often be integrate with your online survey platforms via APIs. They can offer real-time assessments of individual records over a wide range of question types, and can create and export log files and reports. As such, you can report poor quality data back to the sample supplier so they can track which participants consistently provide poor quality data. With better reporting systems, all research buyers end up with better data in the long run.

Digital Fingerprinting

What is digital fingerprinting

Digital fingerprinting identifies multiple characteristics of a research participant’s digital device to create a unique “fingerprint.” When enough different characteristics are gathered, it can uniquely identify every device. This fingerprint can be composed of a wide range of information such as: browser, browser extensions, geography, domain, fonts, cookies, operating system, language, keyboard layout, accelerator sensors, proximity sensors, HTTP attributes, and CPU class.

What types of bad data does digital fingerprinting catch?

• Digital fingerprinting helps identify data from good-intentioned people who answer the same survey twice because they were sent two invitations. This can easily happen when sample is acquired from more than one source. They aren’t cheating. They’re just doing what they’ve been asked to do. And yes, their data might be slightly different in each version of the questionnaire they answered. As we’ve already seen, that’s because people get tired, bored, and can easily change their minds or rethink their opinions.
• Digital fingerprinting also helps identify data from bad-intentioned people who try to circumvent processes to answer the same survey more than once so they can receive multiple incentives. This is the data we REALLY want to identify and remove.

How do digital fingerprinting tools help market researchers?

Many digital fingerprinting tools are specifically designed to meet the needs of market researchers. They’re especially important when you’re using multiple sample sources to gather a large enough sample size. With these tools, you can:

• Integrate them with whatever online survey scripting platform you regularly use, e.g., Confirmit, Decipher, Qualtrics
• Identify what survey and digital device behaviors constitute poor quality data
• Customize pass/fail algorithms for any project or client
• Identify and block duplicate participants
• Identify and block sources that regularly provide poor quality data

Screener Data Quality

In addition to basic data quality, researchers need to ensure they’re getting data from the most relevant people. That includes making sure you hear from a wide range of people who meet your target criteria.

First, rely more than the key targeting criteria – e.g., Primary Grocery Shoppers (PGS). Over-reliance on one criteria could mean you only listen to women aged 25 to 34 who live in New Jersey.

By also screening for additional demographic questions, you’ll be sure to hear from a wide range of people and avoid some bias. For PGS, you might wish to ensure that at least 20% of your participants are men, at least 10% come from each of the four regions of the USA, and at least 10% come from each of four age groups. Be aware of what the census representative targets are and align each project with those targets in a way that makes sense.

Second, avoid binary screening questions. It may be easy to ask, “Do you live in Canada,” or “Do you buy whole wheat bread.” However, yes/no questions make it very easy to figure out what the “correct” answer is to qualify for the incentive. Offer “Canada” along with three other English-speaking nations and “Whole wheat bread” along with three other grocery store products. This will help ensure you listen to people who really do qualify.

Survey Question Data Quality

Once participants are past the screener, the quest for great data quality is not complete. Especially with “boring” research topics (it might not be boring for you but many topics are definitely boring for participants!), people can become disengaged, tired, or distracted.

Researchers need to continue checking for quality throughout the survey, from end to end. We can do this by employing a few more question quality techniques. If people miss on one of these metrics, it’s probably ok. They’re just being people. But if they miss on several of these, they’re probably not having a good day today and their data might be best ignored for this project. Here are three techniques to consider:

• Red herrings: When you’re building a list of brands, make sure to include a few made-up brands. If someone selects all of the fake brands, you know they’re not reading carefully – at least not today.
• Low/high incidence: When you’re building a list of product categories, include a couple of extremely common categories (e.g., toothpaste, bread, shoes) and a couple of rare categories (e.g., raspberry juice, walnut milk, silk slippers). If someone doesn’t select ANY of the common categories or if they select ALL of the rare categories, you know they’re not reading carefully – at least not today.
• Speeding: The data quality metric we love to use! Remember there is no single correct way to measure speeding. And, remember that some people read extremely quickly and have extremely fast internet connections. Just because someone answers a 15 minute questionnaire in 7 minutes doesn’t necessarily mean they’re providing poor quality data. We need to see multiple errors in multiple places to know they aren’t having a good day today.

And of course, if you can, be sure to employ more interesting survey questions that help people maintain their attention. Use heatmaps, bucket fills, gamification, and other engaging questions that people will actually want to answer. A fun survey is an answered survey, and an answered survey is generalizability!

——————————————————–

What’s Next?

Every researcher cares about data quality. This is how we generate valid and reliable insights that lead to actionable conclusions and recommendations. The best thing you can do is ask your survey scripting team about their data validation and digital fingerprinting processes. Make sure they can identify and remove duplicate responders. And, do a careful review of your questionnaire to ensure your screener and data quality questions are well written and effective. Quality always starts with the researcher, with you!

If you’d like to learn how we can help you generate the best quality data and actionable insights, email your project specifications to our research experts using Projects at E2Eresearch dot com. We’d love to help you grow your business!

Learn more from our other blog posts

• Tips for the First-Time Conjoint Analysis Researcher
• 8 Engaging Question Types to Improve Participants’ Survey Taking Experience
• Trackers Suck. Here’s how to fix them.

Research Objective

• A leading bank in the USA needed real-time reporting to enable pro-active, strategic business decisions for a range of audiences in their credit card portfolio.
• The original reporting process resulted in more than 200 pages of metrics and charts, a time-consuming and highly detailed, error prone task for analysts.

Scope & Methodology

• A process that incorporated real-time automation and that used existing software was designed.
• With a combination of routines and systems, the manual report generation process was converted almost entirely to an automated process.

Value Delivered

• The client did not need to make incremental investments in tools. More importantly, the time required by analysts to create quarterly reports was reduced by more than 95%.

Check out other BFSI case studies

• Creating a Financial Portfolio Health Review Report – A credit card business analytics case study
• Customer Behavioral Segmentation to Support Targeted Marketing – A BFSI data analytics case study
• Removing Customer Barriers to Auto Insurance Renewals – An insurance survey case study

Collision 2021 was a four-day, North American tech conference that drew more than 38 000 attendees. I was fortunate to be one of those attendees this year thanks to a ticket kindly donated by ESOMAR. This year, the Collision Conference hosted more than 600 speakers from all walks of life. Just a few of those people included:

• Celebrities: Cindy Crawford, Meaningful Beauty; Maria Sharapova, Therbody; Ashton Kutcher, Sound Ventures; Ryan Reynolds, Mint Mobile
• CEOs and CMOs from global companies: Geoffrey Hinton, University of Toronto; Ukonwa Ojo, Amazon; Fiona Carter, Goldman Sachs; Martin Wildberger, Royal Bank of Canada
• Local and global community leaders: Jagmeet Singh, Leader of Canadian New Democrat Party; John Tory, Mayor of Toronto; Katie Porter, Representative at US House of Representatives; Lori Lightfoot, Mayor of Chicago
• And 13-year-old whiz kids whose expertise and speaking skills rivaled the most experienced speakers in attendance!

With hundreds of sessions running simultaneously (and literally colliding with each other!), it was easy to create a personalized stream of content, particularly since no matter the time, a great talk was always just beginning. The stream I created for myself focused on artificial intelligence, robotics, and innovation. Here are the key themes I took away.

Technology Leaders Must Prove Their Trust

People love their devices. We trust them to help us discover and buy products, make and take phone calls and text messages from our loved ones, and remind us about confidential meetings and doctor’s appointments. We trust our devices will work as expected when we need them to work. However, there is a trust problem and it doesn’t lie with the technology itself. It lies in the fact that we don’t trust the people behind our devices, neither the people building the devices nor our government leaders, to create and hold appropriate boundaries around privacy and security.

Companies build trust by having clear values and a clear mission grounded in being authentic, empathetic, transparent, and relatable. We learn to trust companies that shape our experiences in ways that are personalized but at the same time not creepy. We also learn who trust by witnessing which companies hold themselves fully and immediately accountable when they make mistakes. Companies that abuse these expectations will quickly find themselves speaking to a declining audience. A great way to think about trust is that every interaction a company has with a consumer is either a deposit or a withdrawal. You do good or you do bad. There is no neutral.

Robot, Cobots, and the Inevitable

Did you realize you already have robots in your home? If we follow the strict definition that any automatically operated machine that replaces people is a robot, then your electric toothbrush, your toaster, and your vacuum cleaner (even if it’s NOT a Roomba) are robots. We’re slowly getting used to the idea that robots don’t have to take a human shape to be called robots.

A newer take on robots is the idea of cobots. Unlike a lot of robots that run behind the scenes, collaborative robots are designed to interact directly with or next to people. While you may be nervous that robots or cobots will take your job, there are many good reasons to be excited about working with them. Not only do they easily take on jobs that are dull, dirty, and dangerous, they augment our skills and abilities and help us do our work better and with more agility. Robots make us physically stronger and mentally more agile. If we let them, they help us make truly better decisions.

As in the case of robots and cobots, if something is inevitable, get enthusiastic about it.

The Language of AI

One of the main complaints about artificial intelligence comes when it’s used as a substitute for people. For instance, researchers are actively working on building AI tools intended to serve as personal companions for people who are elderly or disabled, and counsellors for people who’ve experienced trauma. Isn’t that impersonal? Isn’t that disrespectful? Well, let’s consider it from a different angle.

Think about people who’ve experienced a life of trauma, a life wrecked by abuse, trafficking, trauma, or addiction. A life where people have repeatedly let them down and shown that they can’t be trusted. Those who’ve experienced trauma may find it particularly hard to trust new people and may be far more comfortable beginning their healing process by working with AI.

Think about people who have experienced a brain injury or deal with communication disabilities. Or people who aren’t using their native language. Or people who feel more comfortable communicating via email or text. We constantly hear that people should be treated in the way they want and prefer to be treated. That we need to increase accessibility. This could easily be AI.

Regardless of the initial need, we need to ensure that these AI communication tools demonstrate empathy and show respect. AI can’t replace human judgement but it can and should reflect good judgement.

What Does It Mean For Researchers

The research industry talks about trust all the time. We need research participants to trust us enough to share their most personal opinions, their most private click-paths, and their most unusual purchase behaviours. We need research tools that can effectively automate dull and error-prone research tasks leaving us with more time to do our jobs even better and make better decisions.

And we really need to focus on language. So much of our work revolves around language – writing questionnaires with respectful wording that everyone can understand, moderating focus groups that accommodate every participant, making the research space accessible to all.

I may not have attended a single market research talk but I did indeed come away with new perspectives that will make me rethink how I have conducted research in the past, and what I will do in the future.