The Big Game is the biggest advertising event of the year in the U.S., so DISQO put together a comprehensive analysis of the best-performing ads with consumers. After all, what matters most to participating marketers is how the millions they invested moved their brand and their business forward with consumers. These learnings are valuable to all marketers.
DISQO’s Big Game Brand Lift report - linked below - provides quantitative insights on the creative appeal of every commercial, as well as the awareness lift that they provided for their respective brands. Check out this report to learn how DISQO’s leading Brand Lift product and our new Ad Testing product were combined to provide peerless insights about Big Game ad performance with real consumers in our audience.
Our initial report did not cover the thousands of qualitative responses we received from DISQO’s audience members - their verbatim thoughts about why certain ads broke through the pack. Our surveys were completed by over 15,000 consumers, and many left rich comments about the ads that connected (or didn’t). Wouldn’t you like to know what real consumers had to say versus all of the industry’s Monday morning quarterbacks? Here, we examine our rich qualitative data to answer core questions:
- Which ads elicited the most positive comments?
- What themes emerged about the ‘winning’ ads?
- How do the observed qualitative trends compare to our quantitative data about creative appeal and brand lift?
Using qualitative data to supplement quantitative insights
When leveraging unstructured qualitative data like customer quotes about a commercial they saw, its relative importance versus structured quantitative data must be considered.
We collected a huge amount of quantitative data about how different ads fared with consumers (creative appeal and brand lift) and this evidence is the strongest. The insights we harvested are a stable foundation for understanding the performance of every Big Game ad. Because every consumer rated every commercial in the exact same way, we were able to compare ads objectively, apples-to-apples, to determine which strategies and spots won.
However, qualitative data is also valuable and can provide helpful nuance in interpreting the quantitative data. In our initial report, based on quantitative facts, we focused on “what happened,” and now we get into “why.”
- Why did certain ads achieve higher perceptual ratings?
- Why did specific ads elicit higher brand awareness boosts?
- Why were some brands not able to break through on creative appeal or brand lift?
For these types of questions, qualitative data sheds light in ways no other source can. Hearing from consumers in their own words provides a powerful source of incremental information, critical context, commentary, and color.
What themes are evident with the top 25 ads?
To focus our qualitative analysis, we selected the top 25 commercials based on overall performance across creative appeal (i.e., how positive the ad was perceived when watched in a survey context) and brand lift (i.e., how much brand awareness was boosted via Big Game advertising). We then coded the open-ended responses - 100 to 200 comments per ad - into some of the areas discussed below.
Actionability
Approximately 5% of all comments focused on taking action after viewing them. This could mean checking out a movie trailer online, buying a food/drink item at the store, or looking at the brand + its competitors online.
So which ads were most successful generating excited, action-oriented comments? Movie trailers were the clear winners, with Guardians of the Galaxy Volume 3 (20%), Air - Courting a Legend (16%), Indiana Jones - Dial of Destiny (15%), and Creed 3 (13%) all seeing the largest majority of comments focused on action. For newer products, we also saw relatively strong actionability in comments, with brands like Heineken 0.0 (5%) and Google Pixel (5%) influencing consumer motivation toward purchase.
The quantitative data from these ads is fairly consistent with these comment trends. For instance, one of the questions underlying overall ‘creative appeal’ is purchase/action intent (i.e., “This ad made we want to…”). When isolating the ads that drove the highest purchase/action scores (see below), we see many more comments focused on action. In other words, the quantitative results around actionability are strongly related to unprompted comments about taking immediate action. The lone exception to this pattern within our top-5 list below is WeatherTech, which is likely due to their products (car accessories) having a less immediate purchase window.
Theme 2: Celebrity references
Celebrities in Big Game ads is a tried-and-true strategy, but can lead to diverging outcomes depending on how they are used. Simply throwing a few celebrities into an otherwise normal scene can detract from the product message or storytelling approach, undermining creative appeal. However, weaving relevant celebrities into a larger story that accentuates their interests, resume, or talents can help push creative appeal to new heights.
Our audience’s comments show the double-edged sword of using celebrities. For some of our top-25 ads, like DraftKings (25%), Workday (20%), and Dunkin (19%) celebrities were mentioned extremely often. Notably, these brands all leveraged multiple celebrities in their Big Game approach, and this translated into a high volume of celebrity-centric comments involving excitement, humor, and positive sentiment.
However, not all celebrity-oriented comments were positive. The ‘overuse’ of celebrities was mentioned by several consumers and was used to argue against bringing in too many familiar faces. Heavy-handed celebrity endorsements were sometimes distracting for consumers trying to understand the purpose of the ad or the product. The quotes below showcase this mixed sentiment and help explain why our larger report did not find a perfect relationship between celebrity use and quantitative ad performance.
Theme 3: Humor
Sixty-three percent of Big Game ads this year leveraged humor to engage viewers. Some ads that went all-in on humor reaped big responses, like PopCorners, Busch Light, Ram 1500 REV, and Doritos Sweet & Tangy BBQ. For ads like this, humor was mentioned in customer comments at a rate of ~10%.
On the other hand, some ads fell a little flatter (or saw more polarization) over how funny they were. For ads like Avocados From Mexico, E-Trade, and Kia Telluride X-Pro, all of which focused on humor, only ~5% of consumers mentioned that they found them particularly funny. Consumers that found them funny were very outspoken in that conclusion, but others were less enthralled. To ensure consistent humor across an array of target audiences, marketers need to ensure extensive testing is conducted in as many contexts as possible.
Sentiment Analysis and overall ad perceptions
Beyond looking at overall themes, we also coded the sentiment consumers had when discussing each ad. To keep things simple, we coded each quote in terms of whether it was positive, negative, or neutral. Then, we calculated overall sentiment by subtracting the percent of negative comments for any given ad from the percent of positive comments received for that ad. Using this approach, we can quickly see if/how comment sentiment relates to quantitative ad performance on brand lift and creative appeal.
The chart above plots the relationship between comment sentiment and overall ad performance. We see a reasonably strong relationship between these two metrics - a correlation value of 0.17. This suggests that the two values move in tandem more often than not, and highlights the importance that qualitative testing can have on overarching ad performance. However, the statistical relationship here is merely correlational; brands should leverage extensive A/B testing throughout their creative process to more robustly prove cause-and-effect between emotional reactions and downstream impacts.
Consumer quotes that ‘got straight to the point’
Sometimes it’s insightful and fun to simply look at consumer quotes without any filter. If you’ve ever surveyed or interviewed customers, you know few things send a more powerful message than the unambiguous thoughts of your target audience.Below, we share a few direct quotes from consumers about their reactions to some of the highest-performing Big Game ads.
WTH was that about?
One thing we see over and over in the advertising industry are catchy, fun, engaging ads that underplay their association with the sponsoring brand. While entertainment is certainly top-of-mind for Big Game advertisers, this entertainment value can sometimes be over-emphasized at the expense of brand association or a discrete call-to-action. This type of reaction may be one reason why the Tubi ads had a fairly muted impact on brand lift, even though they purchased 3 distinct spots during the game.
Are you seriously going there?
Some ads veer too close to polarizing social, cultural, or political topics. This was certainly the case with the consumer above who thinks that ads on religious topics are too far afield for an event like the Big Game. In fact, these ads from the Servant Christian Foundation received some of the most polarizing commentary of any ad from the game, with some talking about their strength and relevance, and others raising objections for being too politically and religiously-oriented.
Not big enough for the Big Game
With expectations for the Big Game and its commercials extremely high, consumers are judging every commercial much more harshly than they would for a typical programming spot. As such, quotes like the one above are not uncommon for any brand, and are essentially telling advertisers that their effort didn’t have enough ‘glitz and glam’ for their tastes. While understated advertising certainly has a place in the Big Game, some consumers are simply there to be wow-ed.
Learn more in our larger report
The findings here are supplemental to our Big Game Brand Lift report, available for download now. You can dive into:- Which ads performed best on creative appeal and brand lift
- Which strategies led to the most (and least) positive results
- What the key learnings are for brands, agencies, and media companies
Subscribe now!
Get our new reports, case studies, podcasts, articles and events
