Too many children were drowning in Western Australia; the solution was called ‘Every Pupil a Swimmer’ provided training for all children to swim to a basic level of survivability. The solution worked in that the number of children drowning decreased but the solution had a cost, the number of Olympic swimmers for Western Australia dropped significantly. In setting a basic minimum standard there had been an unforeseen consequence of curtailing those with the potential to be Olympians.

I believe the same thing is happening in the world of advertising for major brands: Traditional quantitative online pre-testing of adverts set minimum standards that reject the best creative ideas. This has led to adverts from P&G, Unilever, J&J and other major brands to be survivors rather than Olympians. I’m not alone in this belief, in a meeting with Rory Sutherland he told me:

“(Millward Brown’s) Link testing is effective at getting rid of the worst 20% of adverts but in doing so it also culls the best 20%.”

Rory Sutherland Chairman Ogilvy

Our beliefs are supported by one of the largest studies of creative work in the world ‘Marketing in the Era of Accountability’. In which the respected econometrician found that:

“Campaigns that are quantitatively pre-tested seem to be about half as effective in business terms that those that aren’t.”

Les Binet Marketing in the Era of Accountability

This lack of effectiveness may have played a part in 1,600 P&G staff from non-manufacturing roles recently losing their jobs. P&G CEO Robert McDonald told Wall Street analysts in an interview:

“In the digital space, with things like Facebook and Google and others, we find that return on investment of the advertising when properly designed, when the big idea is there, can be much more efficient.”

Robert McDonald CEO P&G

There are two problems with McDonald’s statement:

Research from the IPA and Deloitte proves that online works best when used in conjunction with TV, indeed TV + the Internet is recognised as a dream marketing combination, not least by the host of online brands who advertise on TV.

Traditional quantitative online pre-testing culls big ideas: In a recent AdMap article Dominic Twose (Global Head of Knowledge Management for Millward Brown) was only able to make a tenuous link between creativity and Millward Brown’s Link test despite give editorial free reign to do so. I’d bet my last £1 that if there was any substantial link between Link testing and creativity that Dominic would have made it clear in the article. Instead one interpretation of the data presented is that award winning adverts are irrelevant and incomprehensible.

Summary

Traditional online quantitative pre-testing inadvertently culls big ideas resulting in ads that are 50% less effective. I believe that is why Olympic quality brands from P&G, Unilever, J&J and many more are surviving rather than winning and without winners more people will lose their livelihoods.

The Pitch

Thanks for reading, now the pitch! If you have ever suffered the pain of failing a quantitative pre-test for a campaign that you just knew was in the top 20% you’ll find working with Bunnyfoot a pleasure: Our research methods are based in the behavioural economics of what people really do rather than what they think and are well suited to pre-testing the top 20% of ideas.
We have completed over 1,300 research projects and Rory Sutherland describes us as “Brilliant!”