You love your ad concept but does your target audience love it?
Advertising is an omnipresent tool that both small and large brands depend on to build and sustain their business growth. But, if your target audience doesn’t like your ads, you won’t be able to achieve the increased brand recall, awareness, purchase, and repurchase rates you were hoping for.
By asking people for feedback about potential ads prior to launching those ads, it’s possible to understand how people perceive them and how they will act as a result of experiencing them.
What are the benefits of ad testing?
Of course ad testing is fun to do but there are important benefits to conducting this research.
Safeguard your brand reputation. Rather than launching an ad that annoys or offends large segments of the viewing audience, conduct ad testing to find and fix unexpected surprises before they become problems in the broader market. Keep in mind that, sometimes, things that are ‘offensive’ to customers or consumers are your deliberate choices and won’t be changed. Either way, ad testing will help ensure the brand is represented as intended.
Use marketing budget wisely. Though conducting ad testing can seem expensive, the potential cost of a failed ad can be vastly higher. Ad testing ensures that ineffective ads are identified and improved before they are launched in-market so that they will generate the awareness or trial as intended. Further, ad testing will help identify in which channels an ad will be most effective. Finding the channel that is appropriate for an ad is just as important as creating a great ad.
Make a good ad great. As one brand among millions, every brand faces fierce competition. Most people see thousands of ads every single day so it’s important to ensure your ad stands out and is memorable in a positive way. Ad testing prior to market launch creates an opportunity to make small or large adjustments so that good ads are even better. Listen to and act on the perceptions of research participants even when they already like an ad. If likeability, uniqueness, or believability could be better, tweak the appropriate components of the ad to improve those metrics.
Understand customer personas. Even great ads are not liked by everyone. Use ad testing to identify which personas prefer which ads so that ads can be presented to the right people in the right channel. If you’ve done segmentation research already, you’ve probably identified 2 or 3 personas that you’ll want your ads to resonate with and you can focus your efforts there.
Make data-driven decisions. Sometimes, multiple ad concepts will seem promising and a clear winner can’t be identified. Ad testing takes the guesswork and personal biases out of choosing a winning ad. And, getting formal approval to run an ad with a big budget is much easier when the data supports the final ad choice.
What research design does ad testing use?
There is no single correct method for conducting an ad test but most use one of three methods, each of which have unique advantages and disadvantages.
Monadic Tests: In this research design, each participant experiences and rates just one ad.
- No order effects. Because people only see one ad, there is no risk that viewing one ad concept before another will have a negative or positive impact on perceptions of the subsequent ad.
- More in-depth results. Because each person spends all their time considering only one ad, you may have the opportunity to ask more in-depth questions about that ad. Alternatively, the questionnaire could remain shorter thereby maintaining engagement throughout the testing period and increasing completion rates.
- Larger sample sizes. You may need a larger sample size to generate reliable data. For example, you might need to sample 300 people for each of 3 ads for a total of 900 people. This could cost a lot more, particularly if your target audience is hard to reach.
- Match group demographics. You will need to ensure that each group is matched on key demographics. For example, you wouldn’t want to discover after the fact that mostly women experienced ad #1 and mostly men experienced ad #2 such that the results actually reflect a gender difference, not an ad creative difference.
Sequential Monadic Tests: Research participants experience and rate several ads individually.
- Smaller sample size: In general, the more concepts each person reviews, the smaller your sample size can be. This is advantageous especially when you must test many ads, and even if each participant only reviews 2 or 3 of those ads. For example, if you’re testing 3 ads and each person sees all 3 ads, you might need only 300 research participants.
- No need to match groups. Because people review several ads, each person acts as their own control group. For the most part, this eliminates the need to precisely match groups based on demographic or psychographic characteristics. This makes sampling much easier and potentially much quicker.
- Order bias must be prevented: To prevent order bias, you’ll need to randomly assign ads to each person, and you’ll need to randomly assign the order in which those ads are presented to each person. For example, if there are 4 ads to review, person 1 might review ads A B and C, person 2 could review ads B D and A, person 3 could review ads D C and E, person 4 could review ads C E and D, and person 5 could review ads E A and B. Order bias will still occur but it will be averaged across all of the ads reducing the potential impact on any one ad.
- Longer questionnaire: Because multiple ads must be reviewed by each person in one questionnaire, it may take longer for them to complete and may result in lower completion rates. Do your best to limit the number of ads each person experiences to no more than 3. And, do your best to keep the completion length under 15 minutes.
Paired Comparison Tests: Rather than showing ads consecutively as in the sequential monadic design, this method has participants experience and rate two or more ads at the same time.
- Smaller sample size: As with the sequential monadic test, having each participant rate multiple ads means you can use a smaller sample size. If you can keep the questionnaire short, this could result in a less expensive project.
- Order effects are reduced. Because people see multiple ads together, the risk of order effects is reduced. Any single ad simply has to be stronger than its competitive ad. Ideally, ads should still be randomly assigned to sets, and sets should be randomly assigned an order.
- Comparisons create cognitive load. Participants may find they like different features of the ads presented to them and consequently struggle to choose one preferred ad. This can cause fatigue, and decrease engagement and attention.
- Longer questionnaire: As before, asking people to rate multiple ads may result in a longer questionnaire. To maintain engagement and completion, you’ll need to take care to ensure the questionnaire is no longer than 15 minutes. You can also help maintain engagement by ensuring you use engaging question types.
What are the key metrics for ad testing?
You could ask each person hundreds of questions about the presentation, messaging, and branding of an ad but you’d end up with a two-hour questionnaire and an unnecessarily low completion rate. Instead, identify the key goals of the ad and choose a set of 3 to 5 key questions that directly address those goals. Then, create a secondary set of questions that will support your KPIs and help you identify points of strength or weakness.
Potential questions could reflect a range of categories such as the following.
Questions about the ad overall
- Overall, what is your opinion about the ad?
- How believable is the ad?
- How relevant is the ad to you personally?
- How appealing is the ad?
- How memorable is the ad?
- Compared to other ads about similar kinds of products, how different is this ad?
- How genuine is this ad?
- What 3 things do you like about the ad?
- What 3 things do you dislike about the ad?
Questions about the presentation of the ad
- What is your opinion about the images used in the ad?
- What is your opinion about the videos used in the ad?
- What is your opinion about the music and sound used in the ad?
- What is your opinion about the spokesperson / mascot in the ad?
- How well does the ad fit into the Facebook environment? Instagram? TikTok? Snapchat?
Questions about the messaging
- What is the main message of the ad?
- How clear was the main message of this ad?
- How did the message make you feel?
- Did you learn anything new from this ad?
- What 2 things did you like about the message?
- What 2 things did you NOT like about the message?
- Which 3 of these words describe how you feel about the message?
- What did the message NOT address even though it should have?
Questions about the brand
- How likely are you to buy this product?
- How likely are you to recommend this product to your family or friends?
- How likely are you to visit the website / download the app / connect on Facebook / enter the draw?
- How much do you care about the brand?
- Did the ad change how you care about the brand?
- Did the ad change how you think about the brand?
How to conduct an ad test
- Identify campaign objectives: Rather than simply entertain, most ads attempt to teach, persuade, or invoke a behavior. For example, teach someone about a product feature, persuade them that a product can meet their needs, or cause them to buy something. Be specific, clear, and realistic about what you expect your ad to achieve.
- Identify the target audience: Consider who you intend to target the ad to. Will it be a broad message intended to resonate with a broad range of people, or a very specific message for specific segments or personas? Have a clear idea about the demographics and psychographics of the audience you intend to appeal to. At the same time, you’ll need to confirm that you can actually reach a representative sample of this audience with your ad. If you can’t reach this audience, you’ll need to identify either a new target group or a new channel.
.
- Identify the key metrics. With the campaign objectives in mind, identify the most important questions that will serve as metrics – your Key Performance Indicators or KPIs. If you’re aiming to build awareness, you’ll want to focus on brand recall. If you’re aiming to invoke a behavior, you’ll want to focus on your call to action, whether that’s purchase intent, connecting on social media, or some other behavior.
- Identify success measures. Once you have your KPIs in hand, you can identify the success measures. This could be to generate purchase intent rates over 50%, brand awareness over 90%, or consideration over 60%. Also, make sure to identify specific criteria for how to differentiate ads that perform similarly. Tie-breaking criteria may relate to superior clarity or believability scores. It’s important to identify your success measures prior to running the test to ensure you stick to your principles of what true success really is.
- Identify the sample size. Once you know how many target audiences you’re addressing and how many ads you’ll be testing, you can determine an appropriate sample size. Remember, the more segments that will be targeted, the larger the sample size will need to be. For a trustworthy outcome, aim for at least 300 people in each segment to review each ad.
- Decide what to test. Ad testing is never an all or nothing decision. Testing could address:
- Several versions of the same ad or several completely different ads
- One or more of the message, image, audio, video, or spokesperson
- General ideas, ad mock-ups, or completely finished ads
- Performance in different channels such as TV, radio, print, OOH, or social media
- Performance against a competitor’s ad
- Performance against an existing or in-market ad
- Create 2 to 4 compelling ad concepts. Though it seems like the first step of an ad testing is building the ads, this can’t be done until decisions are made about the ad’s objective, its intended audience, and its intended channel. With those specific details in hand, it’s time to build a set of ads with a clear message that differentiates the brand and speaks to the target audience. Make sure the ads have a clear Call-To-Action.
- Create data collection tool: Identify the specific questions that will address the KPIs and allow you to evaluate success. In addition, choose a set of complementary questions focused on the various aspects of the campaign that are most important. This could be related to the messaging, the presentation, the brand, or all of those aspects. Remember, for accurate comparisons, all of the questions should be applicable to all of the ads. For example, rather than asking about the piano in one ad and the violin in the other ad, ask about musical instruments or music in general.
Collect and analyze the data. In addition to evaluating the KPIs to identify an overall winner, review all of the metrics to identify any poor performing aspects. Read through the open ends to identify concerns that weren’t anticipated or were left unaddressed in the questions. Further, review the KPIs within each individual segment. It’s very likely that one segment of people prefers a different ad, or that one ad will be more effective in a specific channel. Take advantage of a data dashboard to help find those unexpected insights quickly.
- Fix and retest. A thorough review of the data will help determine whether any aspects of the ads should be corrected or improved before launching. Make those changes and then retest to ensure the ad wasn’t inadvertently weakened. If you’re in a fortunate position that there is a clear winner with minimal to no weaknesses, congratulations!
- Launch the test. In order to ensure the results are as reliable as possible, keep the ad test in field for at least one week. This will give a wide range of people within the segment a chance to participate. Closing field after just a few hours might mean that only desk workers contributed opinions and the results won’t generalize to people who work shifts or who can only check their email once or twice per week.
- Benchmark the results. One of the great things about ad testing is that a few specific KPIs are generally included in every questionnaire. It’s important to maintain a database of these metrics which serves two purposes. First, when calculating KPIs for a brand new ad, it’s possible to compare it to previous ads for other brands you’ve tested. Those ads will likely be associated with in-market purchase data so it will be possible to predict the financial success of the current ad. Second, since the norms database indicates which ads were successful, it’s possible to create new ads that mirror the attributes of those successful ads. In either case, knowing what success can look like means your efforts can be focused in those areas.
What’s Next?
Are you ready to identify your best ads and improve them to match the needs and wants of your targeted audience? Email your project specifications to our research experts using Projects at E2Eresearch dot com. Let’s turn your ad enigmas into ad enlightenment!
Learn more from our case studies
- Benchmarking Brand Awareness and Competitive Brand Space – A consumer food survey case study
- Evaluating the Effectiveness of a Campaign with Brand and Message Recall – An advertising survey case study
- Tracking Customer Satisfaction Across 8 Countries – A food consumer survey case study
- Discovering Brand Awareness to Identify Effective Channel Development | A technology survey case study
A lot of market research starts by truly understanding a specific audience. Who are they – what are their hobbies, where do they live, where do they work, what does their family look like? It’s really easy to calculate a median age and the percentage of customers who are female but the last few years have taught us a lot about intersectionality – it’s not just “women,” it’s “disabled Black women.” In the research world, we understand this as customer segments or personas.
If you’ve written a questionnaire before, you know how important this tip is. Sure, you could write a questionnaire as if you were Charles Dickens showing off his stunning, grammatically correct 200-word sentences with multiple, embedded clauses.
How do you get colleagues to share your research? Easy! Well, it’s not that easy. Storytelling is a necessary skill that will carry your research results throughout the company. Let people know what is exciting about the insights, how they could be used to reach consumers in unexpected ways, how they could personally benefit from understanding the results.
You could build and execute a 5-year research plan.