Request a quote!

Blog, News &

Case Studies

A Practical Guide to Ad Concept Testing
By E2E Research | January 4, 2023

You love your ad concept but does your target audience love it?

 

Advertising is an omnipresent tool that both small and large brands depend on to build and sustain their business growth. But, if your target audience doesn’t like your ads, you won’t be able to achieve the increased brand recall, awareness, purchase, and repurchase rates you were hoping for.

 

By asking people for feedback about potential ads prior to launching those ads, it’s possible to understand how people perceive them and how they will act as a result of experiencing them.

 

 

What are the benefits of ad testing?

Of course ad testing is fun to do but there are important benefits to conducting this research.

 

Safeguard your brand reputation. Rather than launching an ad that annoys or offends large segments of the viewing audience, conduct ad testing to find and fix unexpected surprises before they become problems in the broader market. Keep in mind that, sometimes, things that are ‘offensive’ to customers or consumers are your deliberate choices and won’t be changed. Either way, ad testing will help ensure the brand is represented as intended.

 

Use marketing budget wisely. Though conducting ad testing can seem expensive, the potential cost of a failed ad can be vastly higher. Ad testing ensures that ineffective ads are identified and improved before they are launched in-market so that they will generate the awareness or trial as intended. Further, ad testing will help identify in which channels an ad will be most effective. Finding the channel that is appropriate for an ad is just as important as creating a great ad.

 

Make a good ad great. As one brand among millions, every brand faces fierce competition. Most people see thousands of ads every single day so it’s important to ensure your ad stands out and is memorable in a positive way. Ad testing prior to market launch creates an opportunity to make small or large adjustments so that good ads are even better. Listen to and act on the perceptions of research participants even when they already like an ad. If likeability, uniqueness, or believability could be better, tweak the appropriate components of the ad to improve those metrics.

 

Understand customer personas. Even great ads are not liked by everyone. Use ad testing to identify which personas prefer which ads so that ads can be presented to the right people in the right channel. If you’ve done segmentation research already, you’ve probably identified 2 or 3 personas that you’ll want your ads to resonate with and you can focus your efforts there.

 

Make data-driven decisions. Sometimes, multiple ad concepts will seem promising and a clear winner can’t be identified. Ad testing takes the guesswork and personal biases out of choosing a winning ad. And, getting formal approval to run an ad with a big budget is much easier when the data supports the final ad choice.

 

 

 

What research design does ad testing use?

There is no single correct method for conducting an ad test but most use one of three methods, each of which have unique advantages and disadvantages.

 

Decorative imageMonadic Tests: In this research design, each participant experiences and rates just one ad.

  • No order effects. Because people only see one ad, there is no risk that viewing one ad concept before another will have a negative or positive impact on perceptions of the subsequent ad.
  • More in-depth results. Because each person spends all their time considering only one ad, you may have the opportunity to ask more in-depth questions about that ad. Alternatively, the questionnaire could remain shorter thereby maintaining engagement throughout the testing period and increasing completion rates.
  • Larger sample sizes. You may need a larger sample size to generate reliable data. For example, you might need to sample 300 people for each of 3 ads for a total of 900 people. This could cost a lot more, particularly if your target audience is hard to reach.
  • Match group demographics. You will need to ensure that each group is matched on key demographics. For example, you wouldn’t want to discover after the fact that mostly women experienced ad #1 and mostly men experienced ad #2 such that the results actually reflect a gender difference, not an ad creative difference.

 

 

Decorative imageSequential Monadic Tests: Research participants experience and rate several ads individually.

  • Smaller sample size: In general, the more concepts each person reviews, the smaller your sample size can be. This is advantageous especially when you must test many ads, and even if each participant only reviews 2 or 3 of those ads. For example, if you’re testing 3 ads and each person sees all 3 ads, you might need only 300 research participants.
  • No need to match groups. Because people review several ads, each person acts as their own control group. For the most part, this eliminates the need to precisely match groups based on demographic or psychographic characteristics. This makes sampling much easier and potentially much quicker.
  • Order bias must be prevented: To prevent order bias, you’ll need to randomly assign ads to each person, and you’ll need to randomly assign the order in which those ads are presented to each person. For example, if there are 4 ads to review, person 1 might review ads A B and C, person 2 could review ads B D and A, person 3 could review ads D C and E, person 4 could review ads C E and D, and person 5 could review ads E A and B. Order bias will still occur but it will be averaged across all of the ads reducing the potential impact on any one ad.
  • Longer questionnaire: Because multiple ads must be reviewed by each person in one questionnaire, it may take longer for them to complete and may result in lower completion rates. Do your best to limit the number of ads each person experiences to no more than 3. And, do your best to keep the completion length under 15 minutes.

 

Decorative imagePaired Comparison Tests: Rather than showing ads consecutively as in the sequential monadic design, this method has participants experience and rate two or more ads at the same time.

  • Smaller sample size: As with the sequential monadic test, having each participant rate multiple ads means you can use a smaller sample size. If you can keep the questionnaire short, this could result in a less expensive project.
  • Order effects are reduced. Because people see multiple ads together, the risk of order effects is reduced. Any single ad simply has to be stronger than its competitive ad. Ideally, ads should still be randomly assigned to sets, and sets should be randomly assigned an order.
  • Comparisons create cognitive load. Participants may find they like different features of the ads presented to them and consequently struggle to choose one preferred ad. This can cause fatigue, and decrease engagement and attention.
  • Longer questionnaire: As before, asking people to rate multiple ads may result in a longer questionnaire. To maintain engagement and completion, you’ll need to take care to ensure the questionnaire is no longer than 15 minutes. You can also help maintain engagement by ensuring you use engaging question types.

 

 

What are the key metrics for ad testing?

You could ask each person hundreds of questions about the presentation, messaging, and branding of an ad but you’d end up with a two-hour questionnaire and an unnecessarily low completion rate. Instead, identify the key goals of the ad and choose a set of 3 to 5 key questions that directly address those goals. Then, create a secondary set of questions that will support your KPIs and help you identify points of strength or weakness.

 

Potential questions could reflect a range of categories such as the following.

 

 

Questions about the ad overall
E2E Engame question animation

  • Overall, what is your opinion about the ad?
  • How believable is the ad?
  • How relevant is the ad to you personally?
  • How appealing is the ad?
  • How memorable is the ad?
  • Compared to other ads about similar kinds of products, how different is this ad?
  • How genuine is this ad?
  • What 3 things do you like about the ad?
  • What 3 things do you dislike about the ad?

 

Questions about the presentation of the ad
E2E Engame question animation

  • What is your opinion about the images used in the ad?
  • What is your opinion about the videos used in the ad?
  • What is your opinion about the music and sound used in the ad?
  • What is your opinion about the spokesperson / mascot in the ad?
  • How well does the ad fit into the Facebook environment? Instagram? TikTok? Snapchat?

 

Questions about the messaging
E2E Engame question animation

  • What is the main message of the ad?
  • How clear was the main message of this ad?
  • How did the message make you feel?
  • Did you learn anything new from this ad?
  • What 2 things did you like about the message?
  • What 2 things did you NOT like about the message?
  • Which 3 of these words describe how you feel about the message?
  • What did the message NOT address even though it should have?

 

Questions about the brand
E2E Engame question animation

  • How likely are you to buy this product?
  • How likely are you to recommend this product to your family or friends?
  • How likely are you to visit the website / download the app / connect on Facebook / enter the draw?
  • How much do you care about the brand?
  • Did the ad change how you care about the brand?
  • Did the ad change how you think about the brand?

 

 

 

How to conduct an ad test

 

  1. Identify campaign objectives: Rather than simply entertain, most ads attempt to teach, persuade, or invoke a behavior. For example, teach someone about a product feature, persuade them that a product can meet their needs, or cause them to buy something. Be specific, clear, and realistic about what you expect  your ad to achieve.

 

 

  1. Identify the target audience: Consider who you intend to target the ad to. Will it be a broad message intended to resonate with a broad range of people, or a very specific message for specific segments or personas? Have a clear idea about the demographics and psychographics of the audience you intend to appeal to. At the same time, you’ll need to confirm that you can actually reach a representative sample of this audience with your ad. If you can’t reach this audience, you’ll need to identify either a new target group or a new channel.

.

  1. Identify the key metrics. With the campaign objectives in mind, identify the most important questions that will serve as metrics – your Key Performance Indicators or KPIs. If you’re aiming to build awareness, you’ll want to focus on brand recall. If you’re aiming to invoke a behavior, you’ll want to focus on your call to action, whether that’s purchase intent, connecting on social media, or some other behavior.

 

 

  1. Identify success measures. Once you have your KPIs in hand, you can identify the success measures. This could be to generate purchase intent rates over 50%, brand awareness over 90%, or consideration over 60%. Also, make sure to identify specific criteria for how to differentiate ads that perform similarly. Tie-breaking criteria may relate to superior clarity or believability scores. It’s important to identify your success measures prior to running the test to ensure you stick to your principles of what true success really is.

 

 

  1. Identify the sample size. Once you know how many target audiences you’re addressing and how many ads you’ll be testing, you can determine an appropriate sample size. Remember, the more segments that will be targeted, the larger the sample size will need to be. For a trustworthy outcome, aim for at least 300 people in each segment to review each ad.

 

 

  1. Decide what to test. Ad testing is never an all or nothing decision. Testing could address:
  • Several versions of the same ad or several completely different ads
  • One or more of the message, image, audio, video, or spokesperson
  • General ideas, ad mock-ups, or completely finished ads
  • Performance in different channels such as TV, radio, print, OOH, or social media
  • Performance against a competitor’s ad
  • Performance against an existing or in-market ad

 

 

  1. Create 2 to 4 compelling ad concepts. Though it seems like the first step of an ad testing is building the ads, this can’t be done until decisions are made about the ad’s objective, its intended audience, and its intended channel. With those specific details in hand, it’s time to build a set of ads with a clear message that differentiates the brand and speaks to the target audience. Make sure the ads have a clear Call-To-Action.

 

 

  1. Create data collection tool: Identify the specific questions that will address the KPIs and allow you to evaluate success. In addition, choose a set of complementary questions focused on the various aspects of the campaign that are most important. This could be related to the messaging, the presentation, the brand, or all of those aspects. Remember, for accurate comparisons, all of the questions should be applicable to all of the ads. For example, rather than asking about the piano in one ad and the violin in the other ad, ask about musical instruments or music in general.

 

 

  1. Decorative imageCollect and analyze the data. In addition to evaluating the KPIs to identify an overall winner, review all of the metrics to identify any poor performing aspects. Read through the open ends to identify concerns that weren’t anticipated or were left unaddressed in the questions. Further, review the KPIs within each individual segment. It’s very likely that one segment of people prefers a different ad, or that one ad will be more effective in a specific channel. Take advantage of a data dashboard to help find those unexpected insights quickly.

 

 

  1. Fix and retest. A thorough review of the data will help determine whether any aspects of the ads should be corrected or improved before launching. Make those changes and then retest to ensure the ad wasn’t inadvertently weakened. If you’re in a fortunate position that there is a clear winner with minimal to no weaknesses, congratulations!

 

 

  1. Launch the test. In order to ensure the results are as reliable as possible, keep the ad test in field for at least one week. This will give a wide range of people within the segment a chance to participate. Closing field after just a few hours might mean that only desk workers contributed opinions and the results won’t generalize to people who work shifts or who can only check their email once or twice per week.

 

 

  1. Benchmark the results. One of the great things about ad testing is that a few specific KPIs are generally included in every questionnaire. It’s important to maintain a database of these metrics which serves two purposes. First, when calculating KPIs for a brand new ad, it’s possible to compare it to previous ads for other brands you’ve tested. Those ads will likely be associated with in-market purchase data so it will be possible to predict the financial success of the current ad. Second, since the norms database indicates which ads were successful, it’s possible to create new ads that mirror the attributes of those successful ads. In either case, knowing what success can look like means your efforts can be focused in those areas.

 

 

 

What’s Next?

Are you ready to identify your best ads and improve them to match the needs and wants of your targeted audience? Email your project specifications to our research experts using Projects at E2Eresearch dot com. Let’s turn your ad enigmas into ad enlightenment!

 

 

 

Learn more from our case studies

 

Learn more from our other blog posts

8 (Not-So) Secret Strategies for Great Market and Consumer Research
By E2E Research | August 25, 2021

The secret to successful research may not be a secret but in the hustle and bustle of work, we often forget one or more of them. If that describes your day today, then consider this your quick and friendly reminder!

 

 

#1 Don’t sell: solve problems.

As researchers, our job isn’t to sell questionnaire design, scripting, data analysis, report writing, and dashboards. Those may in fact be the specific services we offer but our real job is help our partners discover practical solutions to their business problems – Why isn’t this SKU selling, what new product do consumers want, who are my customers, how can I upsell to a target audience, how can I complete more projects when half my team is on holidays, how can I help a client when I don’t have all the services they need?

 

Our job is to thoroughly understand the business and research problems, and then translate them into appropriate solutions. Whether it’s concept studies, customer segmentation, journey mapping, market forecasting, or providing professional services, if we can’t translate a need into a custom solution, we’ve not done our job.

 

 

#2 Know your audience

A lot of market research starts by truly understanding a specific audience. Who are they – what are their hobbies, where do they live, where do they work, what does their family look like? It’s really easy to calculate a median age and the percentage of customers who are female but the last few years have taught us a lot about intersectionality – it’s not just “women,” it’s “disabled Black women.” In the research world, we understand this as customer segments or personas.

 

After conducting a well-designed survey, focus group, personal interviews, social listening, or analytics, you’ll have the necessary data to run a reliable segmentation and identify 3 to 5 distinct target groups of people within your ideal audience. For example, a couple of common ones are Primary Grocery Shoppers and Moms of Infants.

 

Once the data has spoken, you can then build a unique buyer persona, a fictional character, for each target group to clearly outline each one’s unique characteristics. This will make developing a set of products, prices, messaging, and marketing that genuinely resonates with each one much easier.

 

 

#3 Map your marketing

Researchers spend a lot of time mapping journeys – shopper journeys, patient journeys, student journeys, employee journeys. Building products that people want to use and buy means understanding the wants, needs, and challenges customers experience at every stage of the journey. You might discover that the most problematic stage, in fact, is not the most problematic stage.

 

Build a plan to understand every stage of the journey from end to end. As eloquently shared by Biz Davis from Abacus Agency, you need to understand whether your brand is lacking in awareness, interest, consideration, purchase, or advocacy, and whether consumers want to be entertained, inspired, educated, convinced, or delighted.

 

 

#4 Think like you search

If you’ve written a questionnaire before, you know how important this tip is. Sure, you could write a questionnaire as if you were Charles Dickens showing off his stunning, grammatically correct 200-word sentences with multiple, embedded clauses.

 

Or.

 

You could search on TikTok and Twitter and find out how people really talk. Use phrases regular people use. Use words everyone understands even if there’s a technically more precise word. Write questions and answers the same way people search and you’ll end up with a questionnaire that people want to answer!

 

 

#5 Promote your content

In the marketing world, this means thinking about native ads, social sharing, and cross-channel marketing. But for researchers, it means sharing your research across the company – from researcher to brand manager to innovation team to development team to marketer.

 

When everyone in the company is familiar with the results of your research, they can each do their part to amplify the outcome of the insight.

 

 

#6 Tell a great story

How do you get colleagues to share your research? Easy! Well, it’s not that easy. Storytelling is a necessary skill that will carry your research results throughout the company. Let people know what is exciting about the insights, how they could be used to reach consumers in unexpected ways, how they could personally benefit from understanding the results.

 

And sure, though the bulk of the research will be educational, informative, and standard, be sure to incorporate just a small bit of fun along the way.

 

 

#7 Become an authority

Don’t rest on the laurels of the research you did last year. That’s old news now. The theory may be correct but times and technology have changed. Follow up last year’s study with one that builds on what you’ve learned from your colleagues, seen among your competitors, and witnessed in related industries.

 

Show your colleagues what your brand could become if everyone worked together to leverage new, innovative research methods, techniques, and skills. Become the expert at your company who constantly pushes everyone forward towards building a better product and a better company. Get that seat at the table.

 

 

#8 Start small to grow big

You could build and execute a 5-year research plan.

 

Or, you could start small with a single project that gives you a solid overview of one product or target audience. Inhale it, memorize it, internalize it.

 

Then build the five-year plan. Because at this point, you’ve seen all the strengths and weaknesses among a specific product, how your colleagues work together, how your company systems work, and what’s happening in your industry. You have perspective now.

 

Now you get it. Now you can think really big.

 

 

My inspiration for this post?

I watched a webinar given by Biz Davis from Abacus Agency in Toronto in which he shared a bunch of his secrets for building an effective marketing strategy. The webinar will be posted on their website very soon so do go have a peek.

 

While watching, all I could think was how relevant his secrets were, in particular, for market and consumer researchers. The headers are his words, and I riffed on the ideas to bring you the research tips.

 

Are you ready to plan a great market or consumer research project from End to End? Email your project specifications to our research experts using Projects at E2Eresearch dot com.

 

 

Learn more from our other blog posts