I always thought Coke and Pepsi tasted pretty much the same.  

When I read Malcolm Gladwell's book 'Blink – The Power of Thinking without Thinking' I realised just how close they once were – but are not anymore. All because of the roller coaster ride Coke took with market testing.

Mini-commercial:
In my previous post about market testing your promotional concept, we looked at the different popular approaches people take to market test their business names or logo designs – without spending thousands on a market research agency.  

The Market Research Challenge

In 'Blink' we see The Coca-Cola Company spending rather a lot on a market research agency in the early 1980s.  Why?  Because that is what large companies tend to do.  But the early 1980s was particularly active for The Coco-Cola Company.  Pepsi was slowly but surely edging in on their market share.  Then Pepsi's sales began increasing significantly when they launched the advertising campaign called 'The Pepsi Challenge'.

The Coca-Cola Company launched into full-scale market research on the matter. Naturally, their testing was focused on comparing Coke to Pepsi. A sample group of dedicated Coke drinkers took a blind taste test sipping each brand.  The majority picked Pepsi as their preferred drink.  

'Eeek!' said The Coca-Cola Company.

Additional tests were carried out and the same results occurred. Pepsi was associated with 'rounded' and 'smooth', while Coke's bite was now being described as 'harsh'.  As a response, Coke released 'New Coke' – a sweeter cola, much like Pepsi.

The disaster

The New Coke sat on the shelves. The product launch was a failure, that much was certain. But The Coca-Cola Company couldn't work out why.  All the testing indicated great success for The New Coke.

The problem with the market testing was that many people found Pepsi nice to sip but were not so partial to drinking a whole can. So – for all the expertise enlisted and money spent – the way testing process was carried out was fundamentally flawed.  Taste testing is fine.  How you taste test is another thing.

A note about focus group failings 

Malcolm Gladwell also cites the focus group failing of successful television shows  All in the Family and The Mary Tyler Moore Show.  

Focus Groups are people gathered by market research companies to talk openly and honestly about how they feel about a particular product(s) presented to them.  This is called 'qualitative' research.  Qualitative research is not designed to find out how many people feel a certain way, but to dig deeper – to discover 'why' and 'how'. 

The third episode of Studio 60 on the Sunset Strip (created by The West Wing's Aaron Sorkin) has in its first scene television network President, Jordan McDeere, pointing out how lousy focus groups are at determining success – listing Seinfeld, Hill Street Blues and (once again) All in The Family as focus group failures. Apparently all these show really did bomb in the focus group tests. 

What's the real problem?

Unfortunately these television show examples have led to high-level criticism of the focus group process.  But, like the Coke example, it is not necessarily the testing model that's the problem.  Taste testing is fine, focus groups are fine.  It is how they are used that is the issue.

Focus groups should not be mistaken for quantitative research. Quantitative research (as in: 'quantity') is for finding out how many will like it.  Alternatively, qualitative research (as in 'quality'), such as focus groups, are great for gleaning insights from your target market to guide your development.

Should market research determine whether you go ahead?

In 'Blink', Gladwell gives another beautiful example of a product that didn't test well.  It was a new ergonomic office chair called Aeron, created by Bill Stumpf of Herman Miller Inc.. Using mesh instead of fabric meant that you could see through to the intricate workings of the design. The comfort level rated through the roof, but the people didn't like the look of it it.  

The company launched the Aeron chair anyway. 

The Aeron attracted great interest from the design industry, won a major design award and the chair became a cult object.  It also began to appear in films and television commercials.  Sales boomed.  What was once perceived as ugly had, with the right exposure, become – as Malcolm described – beautiful.

Perhaps if Herman Miller Inc. had also tested with high level design professionals they would have known that particular group would be the avenue to lead the promotion of the chair.  

It is these kinds of strategies that can come from market testing – that will make the research worthwhile. 

Motto of the story:

When it comes to market testing, be VERY careful with:

• the sample of people you select to test with
• what you show them and/or give them (and how)
• what questions you ask them
• how you interpret the results

Have you done any marketing testing?  If so, how did you approach it?  What happened in the end?  Post us a comment and tell us about the experience.

This post was written by Megan Hills.  Megan is a writer, cartoonist and marketing barista. She also delights in stories about large multinational companies who do silly marketing things, despite their billion dollar budgets. Find out more about Megan.

The last post about briefing your logo designer prompted a great comment.  It was actually more of a query:  

How do you know that your logo is going to appeal to your target market?  

The short answer to this question is: test it on them.

Now, there is testing and there is testing.  You need objective feedback from those who are likely to buy your service or product. But how do you go about getting this objective feedback about your new logo – or any marketing concept?  

Hire a market research agency

You can consider enlisting the help of a market research agency to carry out an investigation on your behalf through surveys and/or focus groups.  The two main areas of market research are: qualitative (why?) and quantitative (how many?).  

If you have a few thousand up your sleeve, then it might be worth getting a quote (or two) from these guys.  Depending on your situation, it could be the best business investment you've ever made for your business.  

But if you're not flush enough for this service (and many in small business aren't), there are other things you can do. 

Let's take a look at some popular common approaches and whether they are useful:

Asking your family

My clients love to test any marketing concept on their family.  Whether it is a business name, logo design or campaign premise, the spouse and little Jimmy seem to always get their two cents.  

Your family are people that already love you and accept you. It's natural to want to involve them.  But don't do it.   For a start, your family probably isn't your target market.   And even if they are, how can they possibly be objective?  

Asking people in your office

Asking your existing customers

This the next thing my clients do.  The idea of asking clients you know really well for their feedback is appealing because, like asking your family, it's not a great imposition.  These are people that have accepted you and wouldn't mind the request.

While we are now in the realm of 'target market' the problem with this approach is that they already know what you are selling.  It is harder for existing clients to pick up on any possible misconceptions.  They will find it difficult to be objective.  

Like with your sales team, by all means get feedback from them.  But don't stop there.

Asking people on the street

My father used to be an executive copywriter for a number of multinational advertising agencies.  While big budgets were allocated to many of his projects, this didn't stop him from going down the end of the street with a clipboard and asking 10 people (in the realm of the target market) three key questions about the product he was working to promote.  

He crafted the questions to ensure that he would not just get a 'yes' or a 'no' answer. Those approached had to explain why they felt the way they did.

At times my father compared the results of this approach with those derived from a market research company.  More often than not the ratio results were the same.  

But your target market might be hard to nab on the street, so be conscious of that limitation.  If your target audience has a broad demographic (both genders and/or large age bracket and/or wide ranging income levels), this way of market testing could be just the thing. 

Asking those you know are potential clients

This is a cracker if you can pull it off.  Is there a positive way to survey those in your target market who are currently buying from your competition?  You are bound to get some interesting data.  However, approach this option with care.  

If the person feels imposed upon by your survey-waving, then the exercise can backfire badly.  You have lost a potential client before you have even started selling to them.  And if they tell other people about their negative experience then you may have lost many potential clients.  

So the ideal is to find ten people who fit your target market profile (that don't know you well – yet) who are willing to give feedback beyond 'yes' and 'no'.

This can sometimes be safely achieved through currently loyal clients who know people just like them.  Again, approach with caution.  Your don't want to spoil a good relationship with a current client.     

In this world of increasing anonymity, people like to give their opinion.  But if it's a ten page survey that is going to take them half a day – with no direct reward – their desire to contribute is likely to fade.

So keep the testing short, make it fun and give those interviewed a reward for participating.

Next post…

..you will find out how even the biggest companies in the world can make HUGE mistakes when it comes to market testing.

This post was written by Megan Hills.  Megan is a writer, cartoonist and marketing cha cha performer.  She's also very nosey about what other people think and feel about things. Find out more about Megan.