Missing the forest for the tests: What the Pepsi-Jenner and Nivea ads teach us about ad evaluation

By Rob Hutton

The recent Pepsi ‘Jenner’ ad and Nivea’s ‘White Is Purity’ campaign are the types of controversies that keep marketers up at night. While media and comedians have dissected the ads, our take is a bit different. As market researchers, who have evaluated hundreds of ads, we have seen our fair share of hits and misses – and prevented a few fatalities along the way.

Our first reaction as market researchers is to ask: “Did anyone even think to evaluate these ads through market research?” If the ads weren’t evaluated, they should have been. Good marketers evaluate every ad in front of the target consumer market. Given the size and sophistication of both brands, it’s probably safe to assume the ads received some review, so we’re forced to conclude that the failure came from the type of review the ad received.

How Ad Evaluation should work:

When I talk about ads, I talk about evaluation – not testing. Testing is about meeting a few key metrics on a tool, but it doesn’t get to the core of the ad, including what’s working, what might be misinterpreted and what could make it even stronger.

When we get ads to evaluate, it is rarely a finished product. We usually get 2 or 3 concepts to evaluate. Some work, some don’t – that’s why agencies typically present the client a few options based on the creative brief. The agency tries to fill that brief creatively, knowing that cutting through with a message today is challenging and needs something that is often a bit provocative or offbeat. It’s not the agency’s fault that some of the concepts might go too far, or that there are inadvertent cultural or social conflicts the ads stimulate. That’s the market researcher’s job – to reach down in evaluating the ads with consumers, and uncover the words, images, icons and hidden meanings that each ad invokes.

How does anyone miss something so important?

It’s hard to imagine anyone missing that “White Is Purity” can and will invoke some pretty strong social and cultural offense. Still, if it was evaluated qualitatively, someone missed it. That is kind of like missing the big red barn door. How could that happen?

Without knowing the specific research conducted, it’s hard to know for sure, but there are a few trends in marketing research that could be at fault. One trend is over-reliance on fashionable algorithms and diagnostic research tools that purport to analyze an ad and supposedly unlock the keys to whether the ad will cut through with the intended audience. There are far too many dubious and unproven claims going around in the race to find an app that is the magic bullet. If this is the culprit, the diagnostics simply missed the obvious because they aren’t geared to uncover these kinds of complex social issues.

If we run the Nivea ad through a variety of consumer diagnostic tools, even ones that measure stimuli in the brain and those that measure emotional responses, the Nivea ad might well pass with flying colors. That is because the large majority of people will see the Nivea message at face value for what it is – simply saying that the color white is a very pure color.  By association the Nivea product gives consumers that same purity for their skin while protecting their clothes. That was surely the intent of the ad, and most will see it that way. But a few will not, and the ad clearly stirred up some deeper feelings. Given the power of social media to spread ideas and outrage, the probably small minority that initially took offence to the ad was able to widely express those feelings and frame the conversation in a very different way than Nivea seems to have intended.

The Pepsi ad is a bit subtler from a market researcher’s perspective. It is very innocuous on the surface. Pepsi did it in-house – and that may have been the first mistake if the producer was also the person overseeing the market testing. Creative projects understandably generate strong emotions with the creator, which is why ad evaluation should be conducted by an unbiased and confident researcher. Even so, having a brand manager separate from the creative director who can take a step back and accept unfavourable findings when they come is vital for marketing success. After all, if marketing research is there to identify problems, then what is the point of evaluating the ad if you won’t accept the findings? And again, even if the ad was tested completely independently, the battery of diagnostics that were used, however good, clearly missed or dismissed the fatal association.

Ultimately, some cohorts of Pepsi’s target audience took cultural and social associations from the ad that have deep emotional resonance and created new, unintended meanings that not only clashed with the intended message, but ultimately created a highly charged negative message.

What should brands look for in ad testing?

In evaluating ads there are many things we look for with consumers, but the four most fundamental questions are:

Is the ad believable?
Is it relevant to the target audience?
Is it unique?
And in the end, does it motivate as intended?

Beyond these questions, a good qualitative researcher evaluating ads should be experienced in semiotics – the meanings of words, iconography – uncovering the cultural or social icons an image may evoke, and an element of cultural anthropology. We have seen, time and time again, a good and creative ad where only one consumer in the evaluation group took a meaning from the ad that popped out as a big red flag.

That is where the Pepsi and Nivea ads appear to have failed in terms of market research. Ads today have multiple meanings – not just a set of diagnostic data. Uncovering those meanings when evaluating the ads, with a knowledge of iconography, semiotics and a keen social and cultural lens, could have prevented both these ads from being run.

Whether these ads hurt the brands in the long term is unknown. We have seen cases where an ad or message creates such strong negative associations that they frame the brand in a way that can take years to unwind. Other times, we have seen an ad or campaign that generates a media backlash where the increased exposure actually boosts short-term sales with no longer term effects.

As always, it is a good idea to find out first, and in the right way.

Robert Hutton is a Vice President at Innovative Research Group, and is an expert in qualitative research techniques and applications.

Looking for advice on your next ad campaign? Click here to connect with Rob.